What I Learned About KPIs from My Six-Year-Old
I arrived to pick up my daughter on the last day of art camp just in time for program evaluations. Since we at the Balanced Scorecard Institute (BSI) use evaluation data for course improvement, I was intrigued to watch a room full of six- to nine-year-olds randomly fill in bubbles and then quickly improve their scores when the teacher noted that if any of the scores were less than three they’d have to write an explanation. 
In the car on the way home, I asked my daughter why she rated the beautiful facilities only a 3 out of 5. She said, “well, it didn’t look like a porta-potty. And it didn’t look like a palace.” She also said she scored the snack low because she didn’t like the fish crackers and wished they’d had more pretzels. As I giggled at the thought of some poor City program planner or instructional designer trying to make course redesign decisions based on the data, I reflected on the basic principles that we try to follow that would have helped the city avoid some of the mistakes they had made.
The first is to know your customer. Obviously, giving small children a subjective course evaluation standardized for adults was ill advised. Better would have been to ask the students about their experience using their language: did they have fun? Which activities were their favorite? Which did they not like as much?
Further, the children aren’t really the customer in this scenario. Since it is the parents that are selecting (and paying for) the after-school education for their children, their perspective should have been the focus of the survey. Were they satisfied with the course curriculum? The price? The scheduling? Would they recommend the course to others?
Another important principle is to make sure that your measures provide objective evidence of improvement of a desired performance result. My daughter’s teacher used descriptive scenarios (porta-potty versus palace) to help the young children understand the scoring scale, but those descriptions heavily influenced the results. Plus a child’s focus on pretzels versus crackers misses the mark in terms of the likely desired performance result.
Similarly, it is important not to get fooled by false precision. Between some participants superficially filling in bubbles and others changing their answers because they don’t want to do any extra work, the city is simply not collecting data that is verifiable enough to be meaningful.
These might seem like a silly mistakes, but they are common problems. We have had education clients that wanted to measure the satisfaction of a key stakeholders (politicians and unions) while ignoring their actual customers (parents and students). We see training departments that measure whether their participants enjoyed the class, but never ask if their companies are seeing any application of the learning. And we see companies making important decisions based on trends they are only imagining due to overly precise metrics and poor analysis practices.
Even the evaluations for BSI certification programs require an explanation for an answer of 3 or less. I wonder how many of our students ever gave us a 4 because they didn’t want to write an answer. I have also seen evaluations go south simply because of someone’s individual food tastes.
At least I can take solace in the fact that no one ever compared our facilities to a porta-potty.
The unspoken implication was that few people ever get significant benefit from their software system, a conclusion I have also observed over my years in strategic performance management.
There are many common reasons that your strategic performance management software system might be getting less use than the gym membership you bought last January. Below are the top 4 that I’ve seen as well as some tips for avoiding them.
Reason 1: You bought into the hype but not the skills
I overheard a CEO recently saying that he needed to buy into the big data craze. It was clear that this person had no idea what big data or predictive analytics meant, but he definitely needed to buy some. Many people seem to think if they just buy some software, within weeks a “number cruncher” will magically come down from a mountain with answers to all of their problems. That is like thinking that if I buy a shovel, a garden will magically appear in my back yard. Performance management and statistical analysis skills are critical to creating value in this field.
Reason 2: You keep the results a secret
The first question some people ask when considering a performance system is, “how do I keep everyone out of my data?” Security around private customer, employee, or some financial information is an absolute must, but a surprising amount of strategic organizational performance information can be shared with leaders and managers. Leaders need information to make decisions and limiting access can communicate that strategy management is something to be left to only a select few. Analyzing data is only the first step. The dialog around why the results occurred and what should happen next are just as critical.
Reason 3: You only use out-of-the-box performance report design
The standard templates provided by the software companies are almost always designed to make the software sell well, as opposed to informing YOUR strategic decision making. Good performance reports communicate three things clearly: 1) How is OUR organization currently performing, 2) Why are WE getting the results that we are getting? And 3) What are WE doing to improve our results?
Reason 4: You count and report on everything that can be counted.
Just because the vendor promises that this tool can handle the volume doesn’t mean that this is a good idea. Strategic performance management is about focusing on the most critical things first. I would recommend selecting a handful of critical performance gaps and focus your data collection, analysis, and improvement efforts on those. Teach everyone in your organization how to do this effectively before you expand to other areas.
There are many more common mistakes, but these four are top of mind for me. Please share other mistakes you’ve seen in the comments section below.
For more about how to improve your performance analysis, see the Performance Analysis chapter of