A recent article in The Wall Street Journal by Khadeeja Safdar and Inti Pacheco, “The Dubious Management Fad Sweeping Corporate America,” discusses the popular method for measuring customer satisfaction called NPS (Net Promoter Score). A number of companies rely on NPS as a primary indicator of the operational health of a company, with some firms even tying NPS to bonuses and executive salaries.

However, NPS is a noisy and volatile metric, and was never intended to drive such decisions. Companies that rely heavily on the metric may be exposed to measurement risk that could lead to poor strategic decision making.

NPS was introduced in 2003 in a Harvard Business Review article, “The One Number You Need to Grow.” Fred Reichheld, the NPS inventor, argued for the power of NPS as a single metric to help companies incorporate customer feedback into company operations. But, as Safdar and Pacheco note, Reichheld is astonished by the misplaced weight some companies put on the metric. “That’s completely bogus,” says Reichheld. “I had no idea how many people would mess with the score to bend it, to make it serve their selfish objectives.”

NPS is derived by asking customers this single: “On a scale of 0 to 10, how likely are you to recommend the company’s product or service to a friend?” This query is then followed by asking the customer to explain their ratings.

NPS divides customers into three groups:

• Promoters (ratings of 9 or 10), loyal customers who will keep buying
• Detractors (ratings of 0 to 6), unhappy customers, and
• Passives (rating of 7 or 8), who are satisfied but could easily be taken by competitors.

NPS is then determined by subtracting the percentage of detractors from the percentage of promoters, resulting in a number between -100 and 100.

NPS has had a surprising run. Consulting firms have always tended to promote managerial techniques that are popular for the season (e.g., Total Quality Management, Six Sigma, Business Process Reengineering), but most of these methods have come and gone. NPS, as Safdar and Pacheco describe has endured and grown from a convenient single number to “a cottage industry of consultants and software firms that help businesses implement and boost their score.”

Despite the popularity of NPS, some academics have questioned its accuracy. Safdar and Pacheco continue, “Two 2007 studies analyzing thousands of customer interviews said NPS doesn’t correlate with revenue or predict customer behavior better than other survey-based matric.” Furthermore, they detail, “A 2015 study examining data on 80,000 customers from hundreds of brands said the score doesn’t explain the way people allocate their money.”

There are other critiques of NPS. Timothy Keiningham, a marketing professor at St. John’s University in New York and co-author of the three NPS studies, notes that NPS lacks peer-reviewed research to support its proponents’ claims of strong correlation with growth. Safdar and Pacheco describe the dubious nature of the calculation: “Some data scientists said the way NPS is calculated, in which one survey metric is subtracted from another, increases the margin of error and requires a larger sample size to get useful results.”

Despite these issues, Bain & Company, the management consulting firm where Reichheld developed the metric, maintains that NPS is accurate and useful. Bain consultants claim that it’s the simplicity of NPS that allows for it to be easily explained to employees, provides an easy way to follow up with customers, and allows for companies to be benchmarked against one another. They also dispute the 2007 studies that downplayed the value of the NPS by claiming that internal analyses of companies are far more effective than third-party analyses.

Some large U.S. firms agree with Bain’s judgment, including IBM, Intuit, and Best Buy. Even within these firms, however, NPS is still questioned. Threads of these discussions can be found on Reddit, for example, where Best Buy employees discuss how to increase the NPS. Their NPS-enhancing strategies go far beyond insuring customers have a great experience. Safdar and Pacheco explain: “They said they can get better results when they explain to customers how the scoring works or tell them their compensation is connected to the result. Some said they only remind the happiest customers to take the survey.”

Despite the fact that so many companies stand by the score, NPS is hard to interpret for investors. Companies typically do not share response rates, margin of error, or whether the results are adjusted for extraneous factors. Safdar and Pacheco comment, “Research shows NPS tends to be higher when response rates are lower, and Americans tend to give higher scores than consumers in some countries such as Japan and Korea.”

In short, companies can use NPS, but it should accompany other more rigorous metrics that accurately measure the state of the customer relationships. The simplicity of NPS allows people to easily adopt the measurement for their organizations. However, the issues associated with it, and the analytical challenges it presents, make it questionable that NPS is the “One Number” you need for customer feedback.