The Ultimate Question 2.0

The Ultimate Question 2.0 is now available! Learn more http://www.netpromotersystem.com

Book-related video

Book trailer

Author Q&A: Why we rewrote The Ultimate Question

Net Promoter Trailblazer videos

Follow us

Sponsored links

« Closed loop feedback videos - by popular demand | Main | Rackspace Net Promoter feedback system achieves >60% response rates »
Tuesday
Feb022010

Relative NPS: How to address the most common NPS misconception

Ron Hollis is the CEO of Quickparts, a company that makes custom-designed parts.  I first learned about them in this Fortune Small Business article last year, highlighting their use of NPS. (The full article is here.)  In addition to founding Quickparts, Ron also wrote a book on design called "Better be Running."
The most common NPS misconception
I recently ran across Ron's blog.  In it, he describes the company's use of the Net Promoter approach to closed loop feedback.  He introduces the topic in a blog post (which you can read in full here). I suspect it is from about a year ago and describes mostly how they calculate the score.  In itself, this isn't all that interesting.  In fact, it appears they use a 1-10 scale rather than a 0-10 scale, and he doesn't make any reference to how they collect the data, from which people in the customer organization they collect the feedback, or what else they ask beyond the simple likelihood to recommend.  So it comes across, if you read only that blog post, as if they just calculate a score and that's all.
He also makes the common mistake of crowing about what we would call "Bottom-up" Net Promoter Score -- collected only from Quickparts customers with a sample and methodology that can't be verified.  Such scores are notoriously unreliable as indicators of a company's overall performance because the sampling methodology relies on existing customers, introducing a bias because it excludes customers who chose not to do business with Quickparts -- or who did business once or twice but were dissatisfied and switched to a competitor.  In some cases, this sample bias is also compounded by how and when the survey invitations are triggered (say, immediately after some sorts of interactions, but not others).
Click to enlargeFar more reliable for comparing scores is what we call a Relative NPS, collected through a "Top-down" process.  You survey a representative sample of all the customers in the market -- whether they do business with you or with one of your direct competitors -- and compare Net Promoter Scores of each relevant competitor.  The "rNPS" for your company is your own absolute NPS minus the NPS of your best competitor.  If your competitor has a higher NPS than you, then your rNPS will be negative.
It is crucial not to stop there.  You MUST also ask why.  You must understand the drivers of difference for your company's performance versus the competition.  The rNPS approach allows you to understand which factors contribute to differences versus competitors and how much.
Calculating Relative Net Promoter Scores requires a little more cost and time, but it's important.  It requires you to be thoughtful about defining the relevant competitive set.  You need an unbiased sample and the sample size needs to be adequate for the cuts you want to make.  You need to survey customers "blind" (without identifying your company as the source of the survey, which can introduce additional response bias), and you need to use a common methodology to compare your company to the competition.
Closing the feedback loop to unlock action
Ron wrote a more recent post, however, demonstrating that they do more than just calculate a score. Click to enlarge This one's a lot more interesting -- you can read it in full here -- because he shares the verbatim feedback from the customer and then describes his company's response.  While this single anecdote doesn't illustrate the full power of a closed-loop feedback like NPS, it at least gives a hint that Quickparts is doing far more with it than just calculating a number and keeping score.  An excerpt from his blog post:
True closed-loop feedback unlocks opportunities to improve individual customer relationships in addition to measuring versus the competition.
QuickParts may be making the mistake of reporting bottom-up NPS externally.  But at least they are creating a true closed loop with customers.

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments

There are no comments for this journal entry. To create a new comment, use the form below.

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>