Have you ever encountered a survey question that asks, “How likely are you to recommend {name of business} to a friend? You probably found an 11 point (0-10) scale that accompanied the question with the endpoints labeled “not at all” and “extremely likely.”

This question is associated with a concept called “net promoter,” and many large corporations are using this as a key metric in lieu of asking a series of more traditional customer satisfaction survey questions. Net promoter (NP) survey questions are designed to measure customer experience and even predict business growth.

Respondents who rate an NP survey question highly — with a 9 or 10 — are called “promoters,” while those who rate from 0-6 are called “detractors.” Those who rate a 7 or 8 are considered “passives.” Some companies will work very hard to try to change detractors to promoters while others focus efforts on moving passives to promoters.

Promoters are loyal customers likely to return and speak highly of the product, service, or business. Passives aren’t as enthusiastic and may easily be lured away by competitors. Detractors (as I’m sure you can guess) can potentially damage business by sharing their negative experiences and perceptions of the product or service.

 

 

 

 

In what seems the zenith of social media, the idea of identifying those who might promote your brand through social sharing channels, and those who might damage it with similar strategies, makes a ton of sense. There’s no shortage of support for employing this “likely to recommend” question in surveys — after all, a large number of Fortune 1000 companies use it — one of which is identifying customer loyalty with a very simple question. The originators of the concept found it worked especially well for mature, competitive industries. But there are also good reasons to consider whether this is the best option for your survey, or whether using differently worded questions might yield better data, especially with certain respondent populations.

My book, Designing Quality Survey Questions, features a number of Real World Questions culled from actual surveys my co-author and I have encountered over the years, and Stories from the Field collected from other researchers and evaluators who shared lessons learned from their survey design experiences. Along with continuing to explore current research on survey design, I keep my eyes and ears open for survey experiences people share with me because these too shape my thinking about quality survey design.

 

 

 

 

Here are a couple of recent examples — real stories, from real people — on this idea of net promoter survey questions:

Story #1: Using NP with children

Prior to our podcast interview, I was chatting with Rebecca from Glass Frog who asked my opinion on net promoter questions and told me a story of her nine-year-old daughter encountering this “likely to recommend” question on a survey about her Girl Scout troop. Rebecca’s daughter provided very favorable responses about her experience with the Girl Scouts. However, when asked whether she would recommend this troop to others, she responded with a 0 on the net promoter question and told her mom, “No. I really like our troop the way it is. I wouldn’t recommend it to anyone else, because I don’t want anyone else to join.” The young girl took the question quite literally and answered honestly. But hey, kids do that, and we know that surveying children has its unique challenges, right?

Story #2: Using NP with seniors

Given my interest in survey design and knowing I was working on the book, my dad, an octogenarian, would occasionally talk with me about survey questions he came across. One day he mentioned the “would you recommend” question he saw on a survey from his insurance company. He said he was troubled by the question, not because he was dissatisfied with the company, but rather because he didn’t see himself as ever having a conversation with anyone about insurance companies. To him, insurance is a private matter, not to be discussed with others, and thus, he felt he simply wouldn’t be in a situation to recommend or not recommend the company. Taking the question quite literally, he didn’t quite know how to answer it. He understood the intent, but was troubled by the particular wording, “would you recommend.”. He’s quite a literal guy, and wanted to answer honestly. He asked me, “Why don’t they just ask me if I’m satisfied with their service? That one I can answer easily.”

 

 

 

 

Lesson Learned:

So here we have it: from eight to eighty, this question is clearly not for everyone! Now, I’m not saying it should never be used. After all, there’s plenty of evidence for the value of the net promoter question to many, many successful companies. My advice is this:

• Carefully consider who your respondents are and how they’re likely to interpret and answer this question;
• Revisit the purpose of your survey (you did articulate a clear purpose in your research or evaluation plan before designing your survey, right?);
• Have a crystal clear understanding of what it is you want to measure — is it customer satisfaction? Is it brand loyalty? Something else?;
• Based on the above, determine what question or set of questions is likely to yield rich, actionable data that will answer your specific research or evaluation question(s); and finally
• Pretest the questions! Use a small-scale pilot test, cognitive interviewing strategies, or other pretesting techniques (all conveniently explained in Ch. 7 of DQSQ!).
• Carefully consider the wording of the main NP question with your respondents in mind:
  • People will interpret these questions in different ways:

How likely are you to recommend [organization name] to a friend, family member, or colleague?

How willing would you be to recommend [organization name] to a friend, family member, or colleague?

• You can also vary the end of the question – “friend, family member or colleague” could simply become “others.” More concise question wording = less cognitive load for respondents and reduces the likelihood of survey fatigue.

• If you use an NP question, follow it up with an open-ended “Why?” question. There’s always a risk that respondents will skip the question or provide non-substantive responses (e.g., “because that’s the way I feel”) but in sifting through these in analysis, you may come across some true gems in the form of highly insightful answers.

NP in the NP world: No easy answers

Non-profits have been experimenting with net promoter survey questions and learning how they can best use them to inform their work. Feedback Labs offers good advice for those wanting to experiment with NP questions including how often to ask, and how to use the data in different ways than corporations do. Others are encouraged by early success with this approach in the social sector, but that said, not all non-profits are in favor of the NP question approach, and this perspective should be taken into account as well.

Tinker, tinker, tinker…

If you’ve read my book or previous articles on survey design you know I’m a huge fan of experimenting with question wording, and a staunch believer that words matter, word choice matters, word order matters, and what you do and don’t ask matters. This is true whether we’re talking about surveys or conversations. After all, how many times have you responded to someone angry or hurt with “but I didn’t mean it like that!”?. If you’re going to try an NP approach, go for it! If you have the opportunity to pre-test with a sample of potential respondents, try some A/B testing, using different versions of the question with random samples of your pre-test respondents. And let me know what you learn!

Many thanks to Chelsea BaileyShea of Compass Evaluation + Consulting, LLC for her generous and insightful feedback on an early draft of this article, and to Rebecca Casciano of Glass Frog for contributing her story.