Ben Satchwell warns that completion rates make capability framework rollouts look healthy while masking failure. Logging in and self-assessing proves compliance, not impact. Instead, measure one behavioural signal that completion can’t fake, e.g. development conversations, hiring decisions, plans that reference capabilities… One honest metric beats dashboards that only flatter activity.
When a capability framework rolls out, the first number anyone reports is completion. 92% of staff have reviewed the framework. 85% have completed a self-assessment against it. The dashboard turns green, the steering group nods, and everyone agrees the rollout is going well. It probably isn’t. Completion is a vanity metric, and treating it as evidence of success is one of the most common mistakes learning and development makes.
You can hit 100% completion in an organisation where the capability framework has changed nothing at all
A vanity metric is a number that looks like progress while telling you almost nothing about outcomes. Completion is the purest example. It tells you people logged in and clicked through. It does not tell you whether anyone understood the framework, agreed with it, or changed a single decision because of it. You can hit 100% completion in an organisation where the capability framework has changed nothing at all.
The same warning applies to competency frameworks and skills taxonomies. Whatever you have built, the proportion of people who “completed” engaging with it is a measure of compliance, not of impact. L&D has a long habit of measuring activity instead of impact: hours logged, courses finished, attendance, completion. These numbers are easy to collect and easy to report, which is exactly why they proliferate. The discipline has known about the problem for decades. Donald Kirkpatrick’s four-level evaluation model, built in the 1950s, exists precisely to push past reaction towards behaviour and results. Most framework rollouts still stop at the first level and call it success.
Why the wrong number wins
Why does completion survive as the headline number? Because it is available on day one, it is unambiguous, and it makes everyone look good. Behaviour change is slow, messy and sometimes embarrassing to measure. Completion is instant and flattering. Given the choice, a steering group under pressure will take the flattering number every time.
The cost is that completion actively hides failure. A framework can hit 95% completion and be quietly resented, misunderstood or ignored, and the green dashboard will tell you none of that. You feel informed while being blind. Worse, you will often celebrate the rollout at exactly the moment it is dying, because the only signal you collected peaks at launch and then goes silent through the months that follow, which is when adoption actually decides the outcome.
Pick one signal completion can’t fake
So what should you measure instead? Pick one behavioural signal that completion cannot fake. Something that can only happen if people are genuinely using the framework to make a decision. The proportion of development conversations that reference a specific capability rather than a generic goal. The number of managers who use the framework to justify a hiring or promotion decision. The share of personal development plans that name a capability gap the framework helped surface. Movement in self-assessed proficiency that a second source corroborates, rather than self-report alone.
Each of those is harder to collect than completion. That is the point. A metric is only worth tracking if it can come back bad. Completion almost never does, which is precisely why it tells you nothing. A behavioural signal can come back disappointing, and a metric that can disappoint you is a metric that can inform you.
You don’t need a balanced scorecard of fifteen indicators. One good behavioural signal beats a dashboard of vanity metrics, because one number that reflects real use will change what you do, while fifteen that reflect activity will only change what you report. Start with the single behaviour that would have to be true if the framework were working, and find a way to observe it.
There is a credibility dimension too. When L&D reports completion to the executive, it reinforces the perception that L&D measures inputs while the rest of the business measures outcomes. Reporting a behavioural shift, even a modest one, moves the conversation onto the ground the business actually respects.
So the next time someone reports that your capability framework is at 90% completion, treat it as the start of the question, not the answer. Ask what changed. Then pick the one behavioural signal that completion cannot fake, and measure that instead. It will be less comfortable, and far more honest.
Ben Satchwell is Head of Capabilities at Acorn PLMS
