Isolating the impact of training

Share this page

Written by Martin-Christian Kent on 12 February 2014

After evaluating your training programme and calculating your ROI, you'll need to report the findings to your various stakeholders. The first question they are likely to ask is 'How do you know that these changes in performance are because of the training?' By evaluating the training at each level listed in the Phillips model, you can show the chain of impact to support your claim that the changes are because of the training. However, what if there has been a venue refurbishment or menu change since you started collecting data? Factors like these show the importance of isolating the impact of the training; otherwise you won't know what the real impact of the training has been without including these other factors.

There are a range of ways to isolate the impact training has on performance. The first step is to identify any other factors that may have influenced your business outcomes; some will be obvious straight away, such as a refurbishment, but others will require further digging. The best approach is to talk to your stakeholders. Those high up in the business will have a good overview of factors that can influence business outcomes, and the people who take part in the training, and their supervisors, will know about any other factors at work. Clients can also help as they may be aware of external factors.

Once you've decided who to talk to, you need to work out the best approach. You can simply send a quick email or questionnaire or, if you have the time, you can interview them. Focus groups can be really useful as they encourage new ideas and can give a broader overview.

When you speak to stakeholders about other factors, ask them to estimate the percentage of change in the business outcomes they feel was due to each factors - including the training. This is called stakeholder estimate, which is fairly straightforward and quick to use. Although the estimates given will be subjective, looking at all the estimates together can give an accurate representation.

Control groups are arguably the most accurate way to isolate the impact of training as a comparison is made between the performance of your training group and the performance of a group that haven't received training. It's important to select a comparison group that is similar to your training group, considering key factors such as location, turnover and other variables that have the greatest influence on performance. However, depending on your organisation, control groups may not be suitable. For example, if there is a bonus scheme related to staff performance, then it's not ethical to withhold training from one group. However, if the training is being rolled out in stages then it may be that a control group is created naturally. 

When using control groups a key problem that can occur is contamination, where the skills or knowledge from the trained group 'leak' to your un-trained comparison groups. To prevent this you can ask the trained group not to share the information, but the most effective way is to select groups that are unlikely to interact. The control group method can seem too scientific for many organisations, but it can give highly accurate information on the isolated impact of training. 

Trend line analysis can be another useful way to isolate the impact of a training programme. Previous performance on an indicative measure is used as a base and a trend line for expected future performance is constructed. Any changes in performance above that predicted by the trend line can then be attributed to the training programme. This method can be used if the trend established prior to training can reasonably be expected to continue if the training hadn't been implemented. The data used to establish the trend line needs to be relatively stable, with enough data to establish a base line (ideally a year prior to the training). This method does assume that other variables that may have affected performance were stable, i.e. were in place both before and after the training was implemented. Trend line analysis is fairly simple to conduct and less time consuming than using control groups and can also be clearly show the benefits. For example, you may be able to show training has reversed a downward trend.

Forecasting is a more scientific approach to trend line analysis, using a mathematical model to isolate the impact of training and other factors that may affect performance. This method can give a highly accurate estimate of the impact of the training, but requires the knowledge and software to complete the analysis. It also can become very complex when there are a lot of potential influencing factors.

Ideally more than one method of isolation would be used to assess the impact of training programmes. However, as seen above, the methods require varying amounts of time and financial investment and businesses will always be concerned with balancing effectiveness with efficiency.

About the author
Martin-Christian Kent is research and policy director at People 1st. He can be contacted via

Related Articles