Heading down a slippery, slippery slope

Should you change the definition of promoters, passives and detractors? Or should you stick with the accepted standard?

This question comes up fairly often in our client work. Many Bain clients are international or global companies. It turns out people in various cultures use response scales differently. Some Asian cultures, for example, mostly use the middle of the scale. Some South American cultures use the extremes. Very few Japanese respondents are give a 10, but Brazilians use the top of the scale all the time.

Should the definitions of promoter (9-10), passive (7-8) and detractor (0-6) be adapted to local markets?

Sony did it!

Click on the image to see full sizeIt appears Sony developed custom definitions of promoters, passives and detractors. I ran across this presentation slide from research firm Synovate in 2009. A client (not from Sony) recently sent it to me again when he wanted advice responding to local market pressure for changing the definition. (Guess which country manager argued for custom definitions. If you think it was the head of Brazil or Mexico, think again.)

I wonder if Sony stuck with their custom definitions, or continued to tweak them. I also wonder whether Sony changed the definitions to suit all the different cultures in which they operate. What's the definition of a promoter for Sony in Germany? Or in Argentina?

Should you customize the definitions?

In a word: no.

But a more sophisticated and complete answer is, as usual, more nuanced than this.

There are three primary reasons not to change the definition of promoters, passives and detractors to reflect the different cultural response styles of different markets:

First, it creates confusion. The Net Promoter system creates a simple, logical and intuitive way of classifying customers: a common language that can be used throughout the organization. The system is defeated quickly when people can't communicate simply and quickly. "Is that a 9-10 promoter? Or is it a 7-10 promoter?" "Oh, yeah. I forgot. In Brazil, only 10s are promoters." Complexity and confusion are the enemies of culture change.

Second, it starts you down the slippery slope of infinite customization. If the team in Japan convinces you 6-10 should be promoters, then shouldn't the team in Singapore be able to convince you to use 8-10? And wouldn't you only classify 10s as promoters in Brazil or Mexico? Some country will argue for a different metric altogether. Head down this path, and there is no way to stop.

Third, it is unnecessary. Comparing Net Promoter scores rarely makes sense across cultural boundaries, making the SCORE comparisons irrelevant. Top-down Net Promoter scores (competitive comparisons based on double-blind research) already take cultural variation into account. Because your customers can only choose from the competitive set operating in their market, it is rare to need a comparison across markets for goal-setting or prioritization. Bottom-up Net Promoter feedback (data from individual customers, whether generated at a touchpoint or on the basis of a relationship) is most actionable at the individual respondent or team level. Comparisons of performance over time are much more valuable than cross-cultural comparisons.

Note that even Sony maintains a consistent definition of promoters, passives and detractors for cross-country comparisons of Net Promoter feedback.

An important caveat

Customer economics -- the basis for cost-benefit or investment analysis -- are based on customer behavior within the market, since behaviors and economics vary. Empirically, it's likely that the behavior of 9s and 10s in a country like Japan would yield quite different economics than the behavior of 9s and 10s in Brazil. This would probably also reflect some important differences in emotional connection, too.

There are two important implications to this differing use of scales across cultures. 

1. Cost-benefit analysis should be done at the country/cultural level wherever possible
2. Individual customer follow-up (closed loop response) should be tuned to the needs of local market customers, based on an empirical evaluation of how customers tend to use the scale

Note: These implications do not pertain only to the Net Promoter system. They are typical of all cross-cultural/global customer feedback systems.

Nevertheless, we recommend keeping the language and framework the same around the world, rather than changing the definitions of promoters, passives and detractors.

But I thought you were a fan of empirical approaches!

The original definition of promoters, passives and detractors was built on empirical observations about the subsequent purchase and cross-sell results with customers plus the successful positive and negative recommendations they made. The break-points in the scale were determined based on natural breakpoints in the data, backed up by qualitative evaluation of the customers' affect when we followed up with them. these break points have been reconfirmed in hundreds of other companies since 2003.

At Bain, we advocate an empirical approach to testing. Do you think your survey should be longer than we recommend? "Well," we will tell you, "test a very short survey alongside a long one. See which one results in the highest quality, most useful responses." Importantly, see which one results in improving response and completion rates over time. Choose the approach that works with your customer base.

But on this issue, empirical controlled tests won't by themselves get you the right answer. The reason: this is not just a statistical issue. It is a change management and human communication issue.

The Net Promoter system is a rigorously practical approach. It is focused on motivating and teaching people throughout the organization to earn the loyal, vocal advocacy of profitable customers. Simple. Intuitive. Common language. Evident linkage between actions and consequences in (almost) real time. In this case, practical considerations trump the analytical and statistical ones. Empirically and statistically, the analysis might lead toward customizing the definition. Practically, however, customizing almost always undermines the entire Net Promoter system and reduces its impact on the organization and its culture.

One final thought

When someone shares their Net Promoter score, you shouldn't need to ask twenty questions to understand what they mean. Especially within a single company. If you're a believer in the Net Promoter system, you are almost certainly a fan of practical, results-oriented approaches to earning the loyalty of your customers and employees. Avoid the confusion, complexity and the slippery slope of infinite customization.

Comments

[Adam Ramshaw]

Rob,

Nice post -- the "what if we just changed the break-point to ..." is a conversation I seem to have with almost every company that starts down the NPS path.

I'm a strong advocate of the standard approach on the basis that the most important reference is not to other companies or even countries but to your own score last week/ month/ quarter/ year. In which case the built in cultural (and other) biases are consistent therefor irrelevant.

Adam

[Willy Azarcoya]

Rob. My name is Willy Azarcoya I am a NPS certified associate. Ive been using NPS for the last 4 years in Mexico. I have found this problem of latin cultures giving good grades and then when we ask why or when we ask some transactoinal questions, the experience is not that great.

There are only two possibilities to avoid this cultural byass. Change the scale or the question.

Ive found a better technique giving them a lot of context before the question. I formulate the question like this.

Imagine a family memeber or a close friend is asking you for advice. He is travelling to Cancun and he knows you have stayed at Fiesta Americana ¿How likely are you to recommend Fiesta Americana to your friend?

This context before the question has helped us a lot to set the mind of the respondent on an "NPS mood" and we have been receiving answers alligned to the experiences.

[Rob Markey]

Willy:

What a great observation! I find the role and impact of including additional context really fascinating as a technique for improving the quality of responses among customers.

Does it also improve the quality of the open-ended, verbatim responses you get to the follow-up question regarding why?

Rob

[Willy Azarcoya]

Rob:

Absolutely, the verbatims are not only more specific, but we have found that they are charged with emotions. And since they feel more engaged with the situation, they give more time to demonstrate why they rate the experience that way.

When we ask why, in order to help them give details and really obtain insights, we ask for examples. Giving context, has helped us to have better access to this examples, I believe those examples are the ones we give when a we are recommending or not a product or service and those conversations are the ones that turn potential customers into customers. It is like when you are selling something, you need to charge your pitch with emotions.

Asking examples has helped us to really go deep and understand what they lived, what they value and how they felt. And since those examples come from what they remember, I feel they are charged with lots of emotions and that WOM of examples is what your brand is able to deliver, at the end yous positioning.

So for me getting the whole package begins with honestly having access and permission of the customer to his experience, and the beauty of NPS is that it goes to the functional attributes of the experience, but since its linked to a person you care about, it goes also to the emotional, so when you give context it really touches those feelings and you can gain access to great information.