Types of KPIs: The Logic Model and Beyond

As part of the KPI Basics series of content we are developing as part of the launch of the KPI.org website, I thought I would introduce the different types of key performance indicators (KPIs). As I describe in the accompanying video, like to use a framework called the Logic Model to describe the first four types. The Logic Model is a framework that is helpful for differentiating what we produce from what we can only influence. It is also helpful for separating between elements that are more operational versus those that are more strategic in nature. For every key process, we spend resources like time, money, raw materials and other inputs. Then every process has measurements that could be tied to that particular process. The outputs of my process are what we produce. Ultimately though, I want to create an impact with my work. Outcomes capture that impact. Let’s look at some examples of these types of measurements in real life. If I am a coffee maker, my Input measurements might focus on the coffee, the water, or my time invested. My Process measures could have anything to do with the process of making coffee, from the efficiency to the procedural consistency. The outputs of my process would be the coffee itself. I could have a variety of measures around the quality of my coffee output. Finally, my outcome measures would be related to things I can only influence, such as if my audience enjoys or buys the coffee. There is certainly more value in measuring impact than there is operations. If my customer enjoys the coffee I am doing something right. But you really do need a mix of both to truly understand performance. To fully understand all of the elements of strategy execution, I can then add a few other broad categories of measures to my story. Project measures monitor the progress of our improvement initiatives and projects and can be designed to improve operations or strategic impact. These track things like scope, resources, deliverables or project risk. In my coffee example, I might have a new branding campaign to sell my coffee. Employee measures tell us if employees are performing well or have the right skills and capabilities needed. I might measure my employees’ skills in making coffee, for instance. Finally, risk measures tell us if there has been an important change in a risk factor that could have a significant impact on our organization. For example, I might have a risk indicator that tells me if global coffee bean availability becomes a problem. The information that these different types of measures provide can be used to inform decision making. Using a family of measure like this can broadly inform your entire strategy. To learn more about Key Performance Indicator development and implementation, please look into one of our KPI training or certification programs or visit kpi.org.
What I Learned About KPIs from My Six-Year-Old

What I Learned About KPIs from My Six-Year-Old

I arrived to pick up my daughter on the last day of art camp just in time for program evaluations. Since we at the Balanced Scorecard Institute (BSI) use evaluation data for course improvement, I was intrigued to watch a room full of six- to nine-year-olds randomly fill in bubbles and then quickly improve their scores when the teacher noted that if any of the scores were less than three they’d have to write an explanation.  In the car on the way home, I asked my daughter why she rated the beautiful facilities only a 3 out of 5. She said, “well, it didn’t look like a porta-potty. And it didn’t look like a palace.” She also said she scored the snack low because she didn’t like the fish crackers and wished they’d had more pretzels. As I giggled at the thought of some poor City program planner or instructional designer trying to make course redesign decisions based on the data, I reflected on the basic principles that we try to follow that would have helped the city avoid some of the mistakes they had made. The first is to know your customer. Obviously, giving small children a subjective course evaluation standardized for adults was ill advised. Better would have been to ask the students about their experience using their language: did they have fun? Which activities were their favorite? Which did they not like as much? Further, the children aren’t really the customer in this scenario. Since it is the parents that are selecting (and paying for) the after-school education for their children, their perspective should have been the focus of the survey. Were they satisfied with the course curriculum? The price? The scheduling? Would they recommend the course to others? Another important principle is to make sure that your measures provide objective evidence of improvement of a desired performance result. My daughter’s teacher used descriptive scenarios (porta-potty versus palace) to help the young children understand the scoring scale, but those descriptions heavily influenced the results. Plus a child’s focus on pretzels versus crackers misses the mark in terms of the likely desired performance result. Similarly, it is important not to get fooled by false precision. Between some participants superficially filling in bubbles and others changing their answers because they don’t want to do any extra work, the city is simply not collecting data that is verifiable enough to be meaningful. These might seem like a silly mistakes, but they are common problems. We have had education clients that wanted to measure the satisfaction of a key stakeholders (politicians and unions) while ignoring their actual customers (parents and students). We see training departments that measure whether their participants enjoyed the class, but never ask if their companies are seeing any application of the learning. And we see companies making important decisions based on trends they are only imagining due to overly precise metrics and poor analysis practices. Even the evaluations for BSI certification programs require an explanation for an answer of 3 or less. I wonder how many of our students ever gave us a 4 because they didn’t want to write an answer. I have also seen evaluations go south simply because of someone’s individual food tastes. At least I can take solace in the fact that no one ever compared our facilities to a porta-potty.
The Ultimate KPI Cheat Sheet

The Ultimate KPI Cheat Sheet

We’ve received a lot of interest in our new KPI Certification Program. In fact, one woman said she couldn’t wait until the first scheduled program offering. She also wanted to know if we had a handy list of the most important principles – she wanted a cheat sheet! So in the interest in tiding her (and others) over, below I have compiled a few of the most important KPI tips and tricks. There are many more of course, so if you think I’ve missed anything, please add them in the comments section below.

Strategy comes first!
A training student told me his organization is struggling to implement measures for brand equity, customer engagement, and a few others because they believed the measures didn’t really apply to their company. I asked him why they were implementing those measures if they didn’t seem to apply, and he said they had found them in a book. They had no strategy or goals of any sort, and yet somehow thought they had a measurement problem.

KPIs found in a book of measures don’t necessarily mean anything in relation to your strategy.  If you don’t have a strategy and/or can’t articulate what you are trying to accomplish, it is too early for KPIs.

KPI Development is a Process
I am embarrassed to admit that the first time I facilitated the development of performance measures with a client, I stood in front of a blank flip chart and asked them to brainstorm potential measures. It was my first consulting engagement as a junior associate and the project lead had stepped out to take an emergency phone call. Even though I had a basic understanding of what good KPIs looked like, I couldn’t help the client come up with anything other than project milestones (“complete the web redesign by August”), improvement initiatives (“we need to redesign the CRM Process”), or vague ideals (“customer loyalty”). What I didn’t understand at the time is that you need to use a deliberate process for developing KPIs, based on the intended results within your strategy. And like any other process, KPI development requires continuous improvement discipline and focus to get better.

Articulate Intended Results Using Concrete, Sensory-Specific Language
Strategy teams have a habit of writing strategy in vague, abstract ideals. As you pivot from strategy to measurement, it is critical that you articulate what this strategy actually looks like using concrete language that you could see, hear, taste, touch or smell. A vaguely written strategic objective like Improve the Customer Experience might get translated into checkout is fast, or facilities are safe and clean. Improve Association Member Engagement might get translated into a result of members volunteer for extracurricular activities. I’ve seen strategy teams shift from 100% agreement on vague ideals to diametric opposition on potential intended results, indicating that their consensus around strategy was actually an illusion.  Use simple language a fifth-grader could understand to describe the result you are seeking. If you spend your time honing this intended result, the most useful performance measures almost jumps out at you.

It’s not about the Dashboard!
Dashboard software is great when it is used to support a well-designed strategic management system. Unfortunately, many people are more interested in buying a flashy new tool than they are in understanding how they are performing (a topic I’ve talked about before). KPIs are not about a dashboard. KPIs are about articulating what you are trying to accomplish and then monitoring your progress towards those goals. A dashboard is the supporting tool and too much emphasis on technology misses and often distracts us from the point.

It’s not about the KPIs!
Speaking of people missing the point, we have many clients who think this process begins and ends with the KPIs themselves. Unfortunately, some of these folks are simply trying to meet a reporting requirement or prepare for a single important meeting. This type of approach completely misses the power of KPI development, which is that KPIs provide evidence to inform strategic decisions and enable continuous improvement.

For more about how to improve KPI development in your organization, see our KPI Professional Certification Program or The Institute Way: Simplify Strategic Planning and Management with the Balanced Scorecard.

Dear Abby-Gail: How Much is Too Much?

Dear Abby-Gail: How Much is Too Much?

There has been a lot of interest in my recent blog post:  “Balanced Scorecard Gone Bad: What’s that Funky Smell?”  Several people have posted comments and questions in various forums, but one in particular deserves special attention.

Dear Abby-Gail: I believe a key point in your message is that a strategy is never static due to external changes (e.g., competitor moves, new technologies), so it will require continuous adjusting.  But this raises a different question. Since as strategic objectives change or the emphasis of what to accomplish within strategic objectives change, this means some KPIs may be dropped and others added (or their weightings may need to be tweaked). As a result, how much change in KPIs can an organization tolerate?

Dear Gary: This is an excellent question.  When strategy changes, then KPIs will have to change. Organizational tolerance to change is affected by several things.

(1) Is the scorecard system engrained in the organizational culture such that management trusts the system and uses it to make decisions?  If so, they will have relatively high tolerance for change in the KPIs because they understand that the change is necessary if they are to continue to rely upon the system to make strategically relevant decisions.

(2) Given that you know you need to adjust the KPI, how quickly can you achieve 7 data points on the new or adjusted KPI?  In other words, is there baseline information available that will help you quickly establish an XmR chart?  If not, can you achieve frequent enough reporting points to have useful trend analysis within 6 months?  If you were using an excellent KPI in the past and then switch to one in which it will be a year (or more) before you have enough data for management to have the 7 data points needed to make statistically sound decisions, this will cause frustration and lower the tolerance for the necessary change.

(3) Can your software system handle these changes without losing your historical performance on the objective (assuming the objective does not change)?  Knowing that you won’t be throwing away historical information increases tolerance for change.

(4) What about rewards tied to KPIs?  How do your Human Resources processes link individual or group performance and incentives to KPI performance?  What will be the result of changing a KPI right now?  If it can’t be changed due to a covenant with employees, can it be removed from the calculation so that you don’t keep working towards an “expired strategy”?

I invite feedback from others.  What else has impacted your organization’s tolerance for needed change in its KPIs?  And does anyone want to share their tips for overcoming resistance to this sort of change?

For more challenges and solutions, we invite you to explore The Institute Way: Simply Strategic Planning & Management with the Balanced Scorecard.

Skinny Jeans and the New Math

Skinny Jeans and the New Math

I am an engineer by training and a math geek at heart.  So articles about girls and math catch my eye.  Did you know that researchers agree that one’s ability to excel at math and science is as much about attitude as it is about “natural gifts” or gender?  This affirms my own less-than-scientific research findings.  I have a daughter and from her earliest years, I showed her how to apply math to everyday activities (baking was our favorite hands-on lesson, of course).  And anytime friends of hers would complain about how hard math was, I’d make them all stand up and shout, “Girls ROCK at math!!!”   It’s all about the attitude.   Of course, I had a good role model for this. My father showed me how fun math was when I was a child as we built motors together and played around with electronics…scribbling equations and schematics as we went.  I never feared math and science…they were FUN!

In my work life, I’ve discovered that dread of math, especially statistics, is widespread in the business community.   So let’s tackle something fun:  the concept of correlation.

When developing performance measures in business, we sometimes face a stumbling block in that the thing we desire most to measure is, unfortunately, impossible to measure directly.  So, we have to look for a “proxy” measure that is correlated.

Let me illustrate with an example from daily life.  Let’s say I want to know if I am maintaining my ideal weight versus gaining weight.  It’s easy to measure that directly – hop on the bathroom scale.  But, unfortunately, I can’t.  I travel constantly so I do not have a bathroom scale with me most days.

So I have a correlate that I measure.  I always carry the same pair of skinny jeans with me.  As long as the jeans will button, I am fairly certain of what the bathroom scale might say, if I had one.  The fit of my jeans is correlated to my weight.   Now, a statistician will remind us that “correlation does not equal causation.”  This simply means is that I need to consider that other things may be causing my jeans not to fit – for example, maybe they shrunk in the wash.  But understanding this, I am reasonably certain that they are a good proxy measure while on the road.

See how easy it was to master two important concepts for measuring performance in business – Direct Measure and Correlated Measure?  It’s all about the attitude!!

To learn much, much more about how to develop meaningful performance measures, we invite you to explore The Institute Way or join us at an upcoming training course.

Free 5-Minute Assessment