I’m continuing to document incremental innovations beyond the net promoter score that occurred in recent years, leading to the current state of fitness-for-purpose analysis techniques. Previously in this series:
- Start with Net Promoter Score
- Add narratives
- Apply segmentation
David J Anderson documented the next two steps in his blog post, Fitness for Purpose Score, so I’m linking to it here as I insert these two steps into the longer story of evolution of these techniques and my experience and perspective on them.
The first of these two steps, which is the subject of this post, was a significant departure from the NPS. The 11-point scale was gone, replaced with a taxonomy of six levels. Some text defined each level, thus providing the customer with options to answer the question about their satisfaction with our product or service.
An important episode from my own experience with the contrast between numeric scales and taxonomies occurred several years to when I became an accredited Kanban trainer (AKT). Early in each class, trainers ask participants to assess their Kanban knowledge and experience. And the way we don’t want to do this is to ask: “Rate yourself on the scale from 0 to 100.” The participant might answer 42 and we’d just wonder what that means. Answers may depend too much not on customers’ real input, but on how they calibrated the scale. Imagine a Kanban user with experience only in a certain flavour of proto-Kanban, which is usually listed third when we cover six proto-Kanban patterns in training. (I don’t have to imagine as I’ve met many such trainees in classes — and the point of the class is to take their game to the next level. The bigger problem than the shallowness of such Kanban implementations is often their not being informed choices — again, another point of taking training is to learn the options.) But they have lots of experience with it, so they may rate themselves 10 out 10 or, allowing for some possibility there’s still something for them to learn in the method, 8 or 9. But from the perspective of a more properly calibrated respondent, they’d certainly belong well on the left side of the scale. Their eight is not another participant’s eight. Thus the numeric scale has too much calibration error and fails to be factual from the point of view of the product vendor or service provider (in this case, me as a trainer) conducting the survey.
Fortunately, even before I joined the AKT program, the community of Kanban trainers had long found a solution to this problem. It was a six-level taxonomy, where substantial team-level Kanban experience was defined as level three. (Definitions of other levels are not the point here, so I’ll skip them. If you’ve taken any certified Kanban training class, you’ve seen them.) I believe this proven solution inspired the six-level taxonomy that appeared in the first Fitness for Purpose survey, which replaced the NPS.
Below is the temporary template consisting of six fitness levels. Temporary, because we would soon adjust its wordings in the very next innovation step. The top two levels mean “promoters”, positive, satisfied customers. The level below them is neutral, and the bottom three levels are for dissatisfied customers with various degrees of dissatisfaction.
- Product/service exceeded customer expectations, delighted them
- Product/service fully met customer’s expectations
- Mostly satisfied, but with some minor concerns or reservations
- Significant concerns or unmet customer needs
- Significantly dissatisfied customer
- Nothing useful, complete loss and waste of time
And, of course, don’t forget to ask the customers to provide a narrative (explain why they chose one of these six answers), and segment the survey by components of the product or service offering.
The next innovation occurred almost immediately after this one, but I’d still save it for a separate blog post. The problem this innovation was to solve was: when the customers choose one of the taxonomy levels, what question are they really answering and what criteria do they apply to judge our product or service? We still had a bit more to go to make survey responses more objective and factual.