Metrics Myth Nine: NPS is Whatever I Say It Is

I’ll just say it right now: I love the Net Promoter Score.  It may not be perfect, but in my experience, it’s the most powerful tool that Service and Support executives have to demonstrate their value to the entire enterprise, become more strategic in how they think about their organization’s mission, and focus the entire company on the customer and their experience.  The NPS is an All Access Pass to the boardroom.

Here’s a brief definition for those unfamiliar with Net Promoter.  Industry research led by Fred Reichheld and his team at Bain & Company determined that customers fall into one of three buckets: they’re promoters who advocate for your brand, detractors who will badmouth you in the marketplace, or neutrals—people who may be satisfied, but who don’t rise to the level of advocate.  If we think of the companies we’re customers of, we can probably easily slot ourselves into one of those three buckets relative to them.

After much quantitative analysis, Reichheld and team determined that the most accurate way of predicting whether a customer is a promoter, detractor, or neutral is to ask them how likely they would be to recommend the company, on a zero to ten scale.  Nines and tens are promoters; sixes and below get classified as detractors; and sevens and eights are neutrals.  If you survey your customers, take the percentage of promoters and subtract the percentage of detractors, you get your Net Promoter Score—that is, the number of promoters net of the number of detractors.  Neutrals don’t count for or against you.  A company that scores all 9s and 10s gets a 100%; a company with all low scores is -100%.  Bain’s research suggests that most companies are in the low positive territory, while leaders can be plus 50% or higher.  (If you’re in negative territory…sell your stock.)

Unfortunately, because of its power, NPS has also gotten trendy.  And trendy measures tend to be misused, or to put it more nicely, “reinterpreted.”  It’s these creative interpretations that are the NPS myth.  Here are some examples I’ve seen:

Redefining the Scale

It’s a small tactical issue, but Net Promoter is based on the zero to ten scale with break points between six and seven, and eight and nine.  I see companies talk about the “NPS,” but pick entirely different scales or endpoints.  There can be good reasons for picking a different scale, but you can’t call what you calculate from it Net Promoter Score.  It’s something different, and the research on NPS may not apply directly, and comparisons with actual NPS scores aren’t valid.

Measuring NPS for Support

The Net Promoter Score is designed to “measure how well an organization treats the people whose lives it affects—how well it generates relationships worthy of loyalty.”  What we’re measuring is the customer’s impression.  And customers don’t care about your internal divisions.  They don’t have one impression of support, another of product development, another of your CFO, and yet another of HR.  By and large, they think about the company overall, and they’ll certainly recommend the company overall (or not.) They can’t say, “Well, buy your accounting software from Company X, but use Company Y’s support organization.”

So why would we measure the Net Promoter Score for Support?  “How likely are you to recommend Company X’s Support Organization to a Professional Colleague?” is an unhelpful question.  It doesn’t work that way.  You either recommend Company X, or you don’t.

As we mentioned earlier, it’s understandable to want to try to measure things that are only under your control, but that’s not how customers experience things, so it’s not a very productive exercise.

Measuring NPS Transactionally

Even more extreme than measuring Net Promoter for Support overall, is measuring Net Promoter for a single support transaction.  It’s a great thing to send (very brief, recurrence-limited) surveys to people who have engaged with your support staff or online resources—but let’s not pretend that their willingness to promote your brand is primarily based on their last support interaction.  Transactional and relationship surveys are different: in transactional surveys, you ask about the transaction, and only in the relationship surveys should you ask about overall impressions.

In transactional surveys, stick to asking about the case, or the web session, and leave it at that.

Compounding this error, I’m sorry to say that I’ve seen companies that selectively measure transactions, exempting entire classes of cases from surveys and giving individuals the option to strike particular troublesome interactions from surveys as well.  Needless to say, there’s enough sample bias and nonresponse bias in most survey programs that adding more with malice aforethought is really a terrible idea, and ultimately self-defeating.  This almost always is a result of focusing too much on the numbers.

Focusing on the Numbers

The root cause for many of these myths is a mistaken belief that the NPS numbers you report, by themselves, mean anything.

NPS, as you’ll recall, was developed as a way of measuring loyalty.  That is, higher NPS scores mapped into higher customer retention, lifetime value, and referenceability.  Since those are future behaviors, they can’t be measured directly, so NPS at its best is a predictor of good things happening in the future.

There is no NPS Bowl where companies or executives get rewarded because their numbers are high.  Whatever credit an executive hopes to get when he presents his “82% NPS” falls apart as soon as a skeptical audience member smells a rat and starts asking questions—at which point, credibility is irretrievably lost.  And, more importantly, although independent research has been mixed in supporting the predictive power of NPS, at least there is some.  Gamed numbers (based on modified scales, limited to support transactions, selectively sampled) have no research supporting their predictive power.

Fred Reichheld himself has said that the important thing about NPS isn’t what the number is; it’s what you do about it.  In the case of Support leadership, it can help line up the entire company on Support’s mission of the customer experience and customer success.  This is a Good Thing.  If we instead create unreliable, Support-only numbers to try to win a nonexistent NPS Bowl, we’ve done our customers, our companies, and ourselves a disservice.