Donald Kirkpatrick (1924-2014)
Richard Griffin reflects on the life and work of the father of training evaluation
Reading time: 7 minutes.
Donald Kirkpatrick died on the 9th May. The words “training evaluation” and “Donald Kirkpatrick” are almost synonymous. Pick up any book on learning and development. If it contains a chapter on evaluating training the chances are the only approach covered will be Kirkpatrick’s now classic four levels model. Why - simply because it is the approach that, now in its fifth decade, not only dominates but has to a large degree defined the field. This really is the measure of Kirkpatrick’s contribution to evaluation, a contribution that began with his 1954 PhD research and the subsequent publication of his model in a series of four articles during 1959 in the ASTD’s US Training and Development Journal. It was not until 1994 though that he published a book based in this research, called Evaluating Training Programs, now in its third edition. He once described how his interest in evaluation began. Regularly teaching seminars to managers he thought ‘why don’t I evaluate them?’
In 2013 Donald’s son Jim, who now runs Kirkpatrick Partners with his wife Wendy, was interviewed for TJ. He was asked why he thought his father’s model still dominated fifty years after it was first published. “People”, he explained, “have told [my father] that what he has done is break down the elusive term ‘evaluation’ into four practical words: reaction, learning, behaviour and results. I think the simplicity of it… has allowed it to remain the premier model”.
Few readers of TJ will need reminding what the levels measure, surely by now they are written into every L&D professional’s DNA, but just in case here they are:
- Level 1, Reactions. At this level the learner’s reaction to the training is gathered and assessed.
- Level 2, Learning. This level is concerned with discovering the extent to which learners have acquired new knowledge, skills or attitudes.
- Level 3, Behaviour. This level is concerned with training transfer – the extent to which the learning acquired in level 2 has been applied in the workplace.
- Level 4, Results. The final level considers the wider impact of the learning
If imitation is the sincerest form of flattery, then Kirkpatrick has been the subject of much appreciation over the years. I recently reviewed the mountain of approaches to training evaluation that have been developed since the Second World War. A truly Herculean task it turned out. There are many, many out there but from Bells System to Xerox, what struck me was just how many were based in one way of the other on the four levels. Most of these approaches represent just semantic changes to the classic model; after 50 years Kirkpatrick’s fundamental insights remain pretty much intact.
A decade ago Jack Phillips suggested the addition of a fifth level: Return on Investment (ROI), the calculation of the cost and benefit of training. ROI has certainty struck a chord with trainers keen to show a financial return on what they do. How easy it is to actually measure ROI is easier said than done though. As a number of commentators have suggested though, ROI is really an amendment to the existing level 4 (results) rather than a new level. In truth there are not many human resource models – and at its heart the four levels is just that, a “model” of workplace learning – that have stood the test of time.
In the late 1980s and early 1990s though a series of academic articles were published that raised doubts about the approach. The basic premise of these studies was that there appears to be no link between responses in one level and results in subsequent levels. If you hand a survey out at the end of a session asking trainees what they think about the session and get good feedback (level 1), do not assume that people have learnt any thing (level 2), or transferred that learning into the workplace (level 3) or made a difference (level 4) because they probably have not. In fact as little as 10 per cent of learning may be transferred into improved job performance and retained over time. The lack of correlation between level results in Kirkpatrick’s model is a problem because almost all training evaluation activity that takes place is based on reaction surveys and in truth it is frequently assumed good reactions do equal impact.
This criticism of Kirkpatrick’s model though is unfair. He always knew what the bottom line was. He told Chief Learning Officer magazine in 2009 that, “unless training gets used on the job, it’s really worthless.” He made it clear in his articles and books that there are four levels in the model and each needs to be addressed. He had previously said that when he started evaluating his own seminars he was tempted just to gather reactions but knew that wasn’t enough. The problem is not the model but the fact that practitioners, for a wide range of reasons, frequently do not go beyond level 1. The latest iteration of the approach developed by Jim and Wendy (called “New World Kirkpatrick”), seeks to address this by focusing practitioners more on the end point of training and stakeholder expectations. The new approach has not yet been subject to academic scrutiny. It should be pointed out that a fair few academics have adopted the Kirkpatrick approach when researching training’s impact.
While he spent almost all his career working at the University of Wisconsin teaching managers, Kirkpatrick worked in industry for four years during the 1960s firstly as training director at International Minerals and Chemical Corps, followed by a stint as a HR director at an aerospace organisation. This work experience, I suspect, was part of the reason for the popularity of his approach among practitioners. There is nothing like the reality of delivering training and struggling with the day-to-day reality of organisations to keep an academic grounded. Whatever the reason, Kirkpatrick created an approach that clearly modelled the training process, provided a map for evaluation but also more than anything else made sense to practitioners – even if often they did not follow it thoroughly. The issue here I think is the need for a greater focus on methods to gather evaluation data that are user friendly but also tell practitioners something reliable about the effects of training.
Training evaluation is a relatively new discipline, particularly compared to the evaluation of learning in schools and colleges (arguably at least three centuries old). Our knowledge of how and why adults learn in the workplace though is growing apace. There have been many insights since Kirkpatrick retired from Wisconsin in 1986. These will shape how training is evaluated in the future. Whatever that future holds the fact that there is an emerging and coherent training evaluation paradigm at all owes much to the foundations Kirkpatrick laid and developed.
Kirkpatrick received many honours in recognition to the contribution he made, including the ASTD Lifetime Achievement Award, ASTD Legend in Training and Development, ASTD Gordon M. Bliss Award and Training Magazine Hall of Fame membership. He was also past president of the ASTD.
Following his retirement, Kirkpatrick wrote a series of books on a range of HR topics including coaching, appraisals, managing change and how to run productive meetings. Outside of his day job, he pursued his love of music (particularly big band and classical) performing with bands and choirs as well as writing songs. In fact when he finally retired from public speaking in 2011 he devoted more time to writing and performing (you can hear the results at http://donkirkpatrick.bandcamp.com). Music aside, he also enjoyed fishing, tennis, golf, as well as being active in his local church.
Writing in January this year, Kirkpatrick reflected on his legacy: “When people think about Kirkpatrick, I don’t want them to think about me; I want them to think about the model and the mission to ensure that training contributes to organizational results. I hope that my model helps to improve training and follow-up so that the lives of those to be impacted by organizations – citizens, customers, patients, clients, children and families – ultimately benefit in some way.”
Kirkpatrick put training evaluation on the radar of organisations and created by far the most popular evaluation approach around; one that has stood the test of time but has not stood still. If a training programme has been evaluated there is a very good chance it’s through the Kirkpatrick model. That in itself is a substantial legacy. His insights though are likely to continue to frame the theory and practice of evaluation for some time yet, helping, as he hoped, to improve training and the impact of training.
Hurix Systems announced today it has been short-listed for Red Herring's Top 100 Asia award, a prestigious list honoring the year’s most promising private technology ventures in Asia.