Erik de Haan made good use of lockdown to take a look at quantitative coaching research – here’s what he’s learnt.
Making an inventory of all quantitative coaching research takes time and through checking a sprawling array of sources and references, 160 independent coaching outcome studies, 35 of which were based on randomised controlled trials, were found and summarised terms of what they show about effectiveness, about the ‘ingredients’ of effectiveness, and about the possible outcomes of coaching [1].
Statistical research is important as it provides the only ‘general’ knowledge about coaching that we have. Any significant patterns found in quantitative research, however small, are predictive and are thus a reliable indication for future contracts and sessions. They can therefore legitimately inform client/stakeholder decisions regarding whether to invest in coaching, and what type of coaching to invest in.
The close examination of these studies have been summarised into five main themes.
Theme 1: mounting evidence that coaching works
Scholars have now gathered enough evidence to confirm that coaching is an effective intervention. Ashridge have undertaken a meta-analysis study comprising 34 of all 35 randomised controlled trials, including seven very recent ones [2].
They found that the effect size of all these studies taken together was 0.62, with a prediction interval that barely includes the zero, so we can say with 95% confidence that coaching will produce a palpable effect over and above a non-intervention control group in exactly the same circumstances (an effect size higher than 0.5 is substantial).
They begin to confirm what many coaches and coachees report from their coaching sessions, namely that coaching offsets the risks of leadership derailment.
It can even be argued that coaching is better researched than a lot of adjacent fields such as mentoring, consultancy, training and management-development programmes – some of which are harder to research because the interventions have fuzzier boundaries.
This is powerful information showing that coaching is worth the investment. It tells us that real, palpable change is likely to result from engagement with coaching.
The only problem with this strong result is that it is mostly supported by the coachee’s self-scores, although if we take the few studies with more objective measures like work results or 360-degree ratings into account, the effect size is still of a similar magnitude.
Therapy has a higher effect size (recent estimate 0.73) [3], but the prediction interval for therapy is slightly lower so the spread of effects is higher.
Theme 2: coaching really flexes around objectives
The second most intriguing finding was how diverse the demonstrable outcomes of coaching were. Although the randomised controlled trial invokes the medical model, coaching does not behave like a simple ‘pill’ that enhances or dampens a very specific function.
Quite the contrary: coaching has been demonstrated to be effective for a wide range of goals, ranging from efficiency savings (call centre times, numbers of units of production per day), to effectiveness improvements (sales conversion rates, service levels, precision surgery), quality improvements (in sensitive meetings, improved health, improved safety standards), and career transitions (higher rates of return to work and of job retention), just to name a few [1].
The most fascinating finding about this ‘range’ of outcomes, is actually the ‘depth’ of the more personal outcomes of coaching, the fact that coaching seems to touch the client deeply: several studies show that coaching has a positive impact on a leader’s personality: assuaging neuroticism, anxiety and fears, impostor tendencies and emotional volatility.
These are still early results, stemming from three recent studies only [1]; nevertheless, they begin to confirm what many coaches and coachees report from their coaching sessions, namely that coaching offsets the risks of leadership derailment.
Theme 3: evidence of meta-understanding between coach and coachee
The next finding is intriguing too. There is evidence in the coaching outcome literature that coaching may achieve the effectiveness that it does, irrespective of the total amount of sessions. Stiles et al. show similar results in psychotherapy, namely that for patients who attend any number of sessions between three and 20, the clinical improvement is broadly the same [4].
What does it mean if coachees achieve the same overall outcome for any number of sessions they negotiate? This could mean that somehow, between coach and coachee, an ‘appropriate’ number of sessions is negotiated for which they will tend to achieve the ‘normal’ outcome (which is that on average a coachee does better than some 72% of the control group).
Or else, that whatever number of sessions allocated to the coachee, coachees will use that precise amount to achieve what they can gain [2]. This is counterintuitive, because normally one would think of the number of sessions as the ‘dose’ of the treatment, and one expects the effect to go up with the ‘dose’ rather than being flat across any number of applied doses.
This finding points at a particular form of ‘placebo effect’, where coach and coachee implicitly negotiate, or accommodate themselves towards, what feels like the ‘right’ amount of sessions. If that continues to be shown to be the case, then it is very heartening for our profession.
It means we can trust our coachees even more than we already do, based on the demonstrated effectiveness, as summarised above. To be precise, we can trust coachees with the ‘number’ of sessions; that they will mostly do well with any number of sessions they agree to attend.
This is a first indication that it is best to leave some of the decision making about coaching with the client. Often the number of sessions is prescribed by the organisation or by the ‘model’ that the coach holds and limited by available budget.
This finding shows that it would be best to communicate to the coachee a maximum number of sessions with no need to use them all. There will be financial constraints at play of course, and there is also a benefit from having a maximised number of sessions as this helps focus the mind on the learning goals.
But most of all, clients may sense intuitively that they do not need the full maximum and can change just as much with a lower number of sessions.
Theme 4: the quality of the relationship matters
All research that explored the quality of the coaching relationship as one of the ingredients of successful coaching, found the relationship to be of great importance. Whether described as ‘rapport’, ‘trust’ or ‘working alliance’, the relationship usually emerged as the strongest predictor of coaching outcome.
We have known this to be the case in the helping professions for some time (see Wampold, for example) [5]. However, it was only very recently that careful longitudinal trials were carried out, where the impact of the coaching relationship was measured from session to session, and the results of these were very surprising [6].
It turned out that although the strength of relationship, that is, the working alliance, correlated strongly with outcomes, there were no significant findings that the alliance also correlated with outcomes per session.
In fact, Ashridge’s research found that there was no link between an increase in outcome after the first session, and any measurements of the working alliance that we did after each of the six sessions of coaching.
This means that working alliance becomes a rather unimportant variable during the work, nothing to do with the impact of the sessions themselves, similar to the coachee’s overall self-efficacy or resilience. Alliance may not even be related to the quality of the relationship as experienced, or as could be observed by an outsider.
This means that we must now go back to the drawing board in terms of measurement of the relationship or of relational coaching as it occurs in the moment. And the link between alliance and outcome has become a lot more tenuous, even controversial.
What we now know about the coaching relationship and how it contributes to results in coaching, should make us think again about the way coaching is implemented in many organisations. Following the research outcomes, it seems really important to allow the coachee to select their coach by having a chemistry meeting or trial session with them.
Not necessarily from a long list of coaches as that may only confuse them: one or two clear recommendations should be sufficient to maximise that overall sense of the relationship which is still a very good predictor of overall outcome.
Coachees should also be allowed to source their coach independently from centralised coaching registers or ‘preferred suppliers’. If they have a personal recommendation from any other qualified coach outside of the organisation’s or provider’s shortlist, then they should be encouraged to work with that coach.
It does not work this way in most organisations today, but this would be a palpable improvement in outcome of coaching, just because the rapport and relationship that clients themselves feel are such an important predictor of overall outcome.
Any coach that they have recruited or have been referred to from their own network, is likely to have higher credibility and reputation with them – and those are other factors that have been shown to make a difference for effectiveness.
Theme 5: the youngest coaches are the best coaches
In the course of my research, I came across a recent article by George et al [7]. who undertook a retrospective study to determine which external coaches of nurses for safe childbirths in India had been the most effective.
Demographics were collected for 10 coaches and (observation-based) practice-adherence criteria for their coachees, by observing 1052 child deliveries. A highly significant (p < 0.0001) inverse relationship was detected between the coach’s age and the nurses’ adherence to the practice checklist.
So, it seems, at least in that special context, younger coaches seem to be more effective than older, more experienced ones. The effectiveness of the coach seems inversely related to both the coach’s age and years of experience, presumably – according to the authors – because young and relatively inexperienced coaches were less directive.
There are so many fresh findings in the literature, and they really seem to strengthen each other. Having summarised all the new data after the recent boom in coaching research and having noted many unexpected surprises in those results, as related above.
Coaching outcome findings generally do not tend to contradict psychotherapy outcome findings [8]. More rigorous outcome studies will appear this year and it looks certain that they will include more information of the relative benefits of virtual and face-to-face coaching, a research topic that is still in its infancy.
About the author
Erik de Haan director of the Ashridge Centre for Coaching at Hult International Business School, UK, and Professor of Organisation Development at the VU University in Amsterdam, the Netherlands.
References
- De Haan E. What works in executive coaching: understanding outcomes through quantitative research and practice-based evidence. London and New York: Routledge; 2021.
- De Haan E. Nilsson V.O. Does executive coaching work? A rigorous meta-analysis based only on randomized controlled trials. Journal of Applied Psychology (submitted 2021).
- Cuijpers P. Karyotaki E. De Wit L. Ebert D D. The effects of fifteen evidence-supported therapies for adult depression: a meta-analytic review. Psychotherapy Research, 2020; 30(3): 279-293.
- Stiles WB. Barkham M. Wheeler S. Duration of psychological therapy: Relation to recovery and improvement rates in UK routine practice. British Journal of Psychiatry, 2015; 207(2): 115–122.
- Wampold BE. The great psychotherapy debate: models, methods, and findings. Lawrence Erlbaum: Mahwah, NJ; 2001; updated 2015.
- De Haan E. Molyn J. Nilsson VO. New findings on the effectiveness of the coaching relationship: time to think differently about active ingredients? Consulting Psychology Journal 2020; 72(3): 155–167.
- George E R. Hawrusik R. Delaney M M. Kara N. Kalita T. Semrau K E. Who’s your coach? The relationship between coach characteristics and birth attendants’ adherence to the WHO Safe Childbirth Checklist. Gates Open Research, 2020; 4(111): 111.
- De Haan E. Relational coaching: journeys towards mastering one-to-one learning. Chichester: Wiley; 2008.
