Training evaluation in this new world
Last month I blogged about learning being a sacred cow, and that the emphasis should be on performance as the outcome rather than learning as the outcome. So I was most intrigued to have the opportunity to attend a short course on the Kirkpatrick "New World" Four Levels model run by DPG a couple of weeks ago.
Given my proactive stance on informal learning in all its guises, and my lack of enthusiasm for much of the formal training that currently gets delivered, going to a formal training course on how to evaluate training seemed like a strange thing to do. I suspected that I would spend a lot of my time biting my tongue to quell my objections, or squirming in my seat as I disagreed with those around me worshipping at the Temple of Training.
I was wrong.
I knew something about the model, or at least I thought I did. Like many in the learning community I knew it was an old and venerable model for measuring learning at different levels. The first level being the happy sheet, and easy to do; and the higher levels progressively more difficult to do and therefore often not done. Nice in theory, but not so easy in practice.
I was wrong again.
What turned things on their head for me was how the model has evolved since it was first published in 1959. That is a very long time of continuing development in response to business needs and the changing business environment. My knowledge of the model, what it actually is and how to use it was clearly well out of date.
What got my attention very quickly was the use of the Stephen Covey catchphrase, "start with the end in mind". According to the New World Kirkpatrick Model, the very first thing to do before you have even confirmed that training is indeed a possible solution, is to focus on the business outcomes that are required. Given I have said this numerous times and even published it in my little booklet on tips for L&D people, all I could say was "I agree, this is good thinking".
And I could also see around me the internal paradigm battle that often occurs when you get people in L&D to focus on business outcomes without thinking about training at the same time. Typically, given a brief sniff of a business goal, they are quickly off in their minds designing some new training intervention based on what they know how to do.
Then we talked about the critical behaviours that would need to be in place in order for this business outcome to be realisable. What do people need to do, how do they need to perform, how do they need to behave in order for this business outcome to happen?
Again, this is something I have said many times. By now I had stopped biting my tongue and squirming in my seat. I felt right at home with the thinking and was looking forward to what the rest of the course had to bring.
They did indeed cover all four levels of the model but I sometimes wondered if they regretted the numbering system that has been applied to them. The reason for this is that the fourth level is about business impact and this is where you start. And you do this before you have even thought about your training course, who needs to be trained, where you might hold it, how many days it will be, what content it will have and how you will blend it with other delivery channels.
As someone who has operated at director level in a NASDAQ quoted global company, I could really appreciate this. When people in the teams I was responsible for went on training courses, I was largely unmoved by what they thought of the cheesecake they had for lunch, or even what they had learned. I was mostly concerned if they could improve their performance, and thus the KPIs of the areas in the business I was responsible for.
Mind you, the cheesecake I had for lunch at this Kirkpatrick training course was a fine example of culinary art. It was blueberry cheesecake (and I put cream on top).
I came away with the realisation that the Kirkpatrick model can be wrapped around a training course. It guides the thinking you need to do before developing or buying your training, it guides the thinking for the design of the training, it guides the evaluation of the training event itself, and then guides you through the evaluation of the effectiveness of that training in terms of its impact on the business. It is virtually impossible to use Kirkpatrick evaluation successfully after a training course unless it was part of the thinking when the training course was just a twinkle in your eye. But if you do use it at the twinkle stage, it is relatively easy to put in place, and greatly helps with getting the right training to the right people, and with tools in place to prove that to the business.
There is a great deal of difference between effective training and training effectiveness.