Different ways to measure the effectiveness of talent management interventions
In my September blog I discussed effective talent management strategies. One of the key things you need to consider when implementing any talent management strategy is whether it is likely to lead to a return on your investment.
There are a number of ways you can look at this - perhaps the most commonly known methods are the Kirkpatrick or Phillips methods of training evaluation. However, there are also less well-known methods that can prove useful when conducting an evaluation of training that takes place on the job, or more ‘soft’ interventions.
Brinkerhoff proposed a Success Case Method in 2003 which, as the name suggests, looks into individual cases and builds stories of the impact of an intervention. There are five key steps involved:
- Identify what the goals of the intervention are and outline what success looks like to you
- Survey a large number of your participants to identify the most and least successful cases
- Gather data on the factors that led to their successes and failures – both enabling factors and obstacles
- Analyse the data and put together a story for each case
- Share the stories and use the findings to educate the organisation.
Examining the most and least successful cases in an intervention can help you to understand the root cause of success and failure. The aim is to examine these stories thoroughly enough that the evidence is strong enough to “stand up in court”.
Another method that can be useful when considering the evaluation of a more ‘soft’ intervention is Theory-based impact evaluation (TBIE) which looks to understand the reasons for an effective intervention and the circumstances under which results are likely to be replicated. TBIE does not prescribe specific methods to use, but suggests guidelines for a valid and reliable evaluation.
There are six key principles to the approach:
- Map out the causal chain from the intervention to the desired outcomes
- Understand the context, i.e. the social, political and economic setting where the intervention is taking place
- Anticipate that success of the intervention can vary according to a number of factors
- Evaluate of the impact of the intervention using a credible control or baseline comparison
- Assess the assumptions that occur in the causal chain
- Use both quantitative and qualitative methods to get a broader overview of the impact.
Regardless of which method you use to evaluate an intervention it is key to start with the end in mind – i.e. does the intervention really target the problem area you are trying to address? What will success look like to you?
Also, it is important to consider the reasons why you need to complete an evaluation, as this will help shape the method you use. For example, if the purpose is to demonstrate the effectiveness of the intervention to your CEO to secure future funding, the Phillips method which can demonstrate a ROI figure may really support your case.
Yash Dubal guides you through the immigration visa maze.
Charles Hipps tells us why recruiting for diversity benefits everyone.
In the first three parts of a new set of mini-guides called TJ Essentials, Steve Macaulay and Sarah Cook take a look at change management.