Lisa Sofianos challenges our thinking on evaluating coaching
Just for fun, I have completed a trawl of the web on the words ‘measuring coaching impact’, and what I have found worries me. What I have been met with is a bewildering and complex fug of opinions, strategies and byzantine methods that are all engaged in the sincere purpose of offering clarity around the value that coaching delivers. I don’t doubt their intent or the amount of thought that has been invested in their design but the sheer quantity of approaches and seeming lack of consensus around the best way to measure coaching impact is troubling.
I understand the dilemma for organisations that have invested large sums of money into coaching programmes and feel the pressure to provide hard evidence to demonstrate that their money has been well spent. And as a coach myself, I share the desire to be able to point to some figures that persuade irrefutably the effectiveness of coaching in the workplace. But what I see in almost all of these proposed methods for measuring effectiveness is anxiety around coaching as a developmental method in general and a profound confusion over how to categorise and anatomise it.
Facing some realities about the value of coaching: No.1
In an effort to cut through this confusion and anxiety let’s face a few realities. Before we start to address the return on investment question, the value for money issue, or seek to determine how much better coaching approach A is to coaching approach B, we need to ask of ourselves a broader question and that is: “Am I persuaded that there is value in coaching?” The reason why this is a first order question is that so many organisations seek to measure the impact of particular coaching assignments when what they are often doing is implicitly putting on trial the coaching discipline as a whole. It seems to be a rather big ask of any coach to simultaneously do their job and defend the honour of their entire profession at every outing, and this is certainly what I see happening.
Will, skill and drill
So let’s start with the bigger question by looking at what coaching actually entails. Coaching is a process that (typically) two people deliberately and willingly enter into, in order to work on real, important and complex workplace challenges. One individual (the coach) guides the other (the coachee), in a skilled way to help them understand the issue (or opportunities) that they face and to dig beneath the presenting problem to develop a capability in addressing their challenges.
This is undertaken within an accountability framework (a process referred to as ‘contracting’) whereby the coachee agrees to be held responsible for making progress against agreed actions and this becomes an anchoring feature of the relationship. All of this is done by applying the best available knowledge and insights from the fields of psychology, social psychology, organisational theory and the professional experience of the coach.
With this in mind, it would be hard to argue that such an approach would be unlikely to deliver value, in fact the trinity of will (of the coachee to make change), skill (of the coach) and drill (generated by the process) are nothing short of a golden recipe for success and incidentally are rarely all present in other forms of development intervention such as leadership development.
Facing some realities about the value of coaching: No.2
The second reality that we need to face relates to some fundamental assumptions about coaching impact measurement, and in particular, that a physical science framework is entirely inappropriate as a methodology for examining a social science process.
Right ambition, wrong tool
Students of the physical sciences like: physics, biology, chemistry and so on, can assume certain constants in the behaviour of the objects that they are studying. It is possible to control variables of weight, size, texture, temperature, volume and pressure in order to isolate certain results and draw reliable conclusions. Scientific conditions are controllable and reproducible so that hypotheses can be tested and results compared. We know that if you add heat to water it will boil after a predictable amount of time. If you apply pressure to glass it will shatter, and so on. Cause and effect in the physical sciences is demonstrable exactly because of the fixed properties that objects possess.
However, the minds of human beings do not act like atoms or molecules that respond predictably to pressure, heat or other outside stimuli. Furthermore, no two people will react to coaching in the same way, despite the fact that they have used the same coach, for the same number and frequency of sessions and for ostensibly the same issue. It is unhelpful to construct systems of measurement that make the assumption that they do. Coaching works on our sense-making capabilities and so a measure of its efficacy must surly start with, and focus mainly upon, the individual and what has changed for them. We can of course ask others around them to describe any changes that they observe but even this is flawed to some extent. For example, the reason why Bob is observed by his colleagues as being less stressed could just as much lie with the fact that he is sleeping better on a newly purchased mattress as it does with the coaching he has just received.
And so, in the physical sciences we can conclude that action A causes reaction B to occur, based on an understanding of the properties of A and B. However, in the social sciences, we must recognise that nestled between A and B is a process of sense-making that is undertaken by the individual, in this case the coachee. It is this sense-making capability that in every way differentiates us from the atoms and molecules around us and explains why value assessment methods must be different from the physical science discipline; falsely scientific claims of the proven value of coaching have to be tossed out of the window if we are to make real progress. When it comes to the question of who should be the arbiter of value, it is clear that the coachee simply has to be firmly set centre stage.
Facing some realities about the value of coaching: No.3
A third reality that we need to face is that we get what we measure rather than measuring what we get. Or to put it another way, David Cooperrider’s brilliant observation that “Human systems grow in the direction of what they persistently ask questions about” puts it very well. If we focus all of our efforts on measuring the objective effects on organisations, we could be running the risk of poorly targeting coaching interventions and ignoring or diminishing the individual benefits to the people who are experiencing the coaching. There is a disconnect here between what coaching achieves well and what we choose to value within it, and we need to have a proper conversation about the unique place of executive coaching in the range of development tools as a starting point of any method for establishing its impact.
Here is an example to illustrate the point. In order to deal effectively with organisational uncertainty, executives need to be able to stand back from the day-to-day turbulence of corporate life and engage in the thinking necessary to appreciate and react to emergent and unpredictable issues. This understanding sits at the core of the executive coaching process. Not to mention that effective leadership of others requires the leader to have good personal insight and awareness of the impact of their personal thoughts, feelings, and behaviour. We know that coaching works on increasing such insight.
However, a focus on objective organisational effects cannot begin to fully engage with these changes in mind-set and thinking process, some of which take time to emerge, and so ignores most of the value of the coaching in this respect. How one would go about measuring the effects of coaching on, say, wellbeing issues like stress and executive burnout using such a framework is also open to question.
What does good coaching looks like?
Once we release coaching from the notion that it is on trial, and we choose not to be bogged down by the false assumption that it can be measured in the same scientific manner that we measure physical phenomena, we can then move to some more useful questions such as “what does good coaching look like?” One way of answering this question is to look at what we would hope to be present in any coaching situation in order to deliver real value. You might think of this as
a recipe.
Required inputs:
- Those engaging in coaching are informed by high quality academic and professional thought and research
- The coach is appropriately skilled, experienced and professional
- The coach is affiliated to a body with a required code of practice
- A coachee is genuinely committed to undertaking the work necessary to make the process a success
- A successful ‘chemistry’ meeting between the coach and coachee takes place at the outset
- Specific and well structured coaching ‘contracts’ (in the psychological contract sense) exist between all parties
- There is a work environment that offers support to any consequent changes in the coachee.
Required outputs:
- The coachee judges the coaching to be effective and to deliver value
- An observable positive change in the actions/conduct of the coachee occurs
- The contracted goals agreed between parties are met
- All parties involved in the coaching relationship are engaged and informed throughout the process.
In addition to these elements of the coaching ‘recipe for value’, we might also look out for particular ‘stretch’ factors.:
Stretch factors:
- The coachee reports changes occurring at a deep level, such as being able to successfully make changes to issues they have struggled with (or been ‘blocked’ on) for some time
- The coach reveals to the coachee certain realities (for example how they are seen, where their blind spots lie, what their strengths are and so on), that were not properly appreciated by the coachee before the process started
- The coachee learns from the process about how they themselves can enquire differently about complex and difficult issues. In this sense the benefits are not just issue specific but have sustainability
- The coachee learns to take full responsibility for their learning and improvement agenda.
Taken together, these inputs, outputs and stretch factors take the conversation about coaching value to a much better place.
Coaching is valuable, but how valuable?
The second order question that follows whether coaching is valuable is the question of “what is its worth?” And this is a hard question to answer because it depends on factors such as the specific provider that is under the microscope, and the specific coachees in receipt of coaching – as the coachee is as much a part of the value chain as
the coach.
But at a more fundamental level, it also depends on the measure of value you would find acceptable and the position you take on who should be the arbiter of value. These are all powerful questions, and if we are to attempt a traditional return on investment (ROI) calculation then we would need to have them satisfactorily answered – which I would submit is very hard indeed. For example, some people have sought to suggest that coaching has given rise to positive financial benefits in the company such as increased revenues, higher sales and so on.
Others have argued that coaching has raised the levels of confidence for example of senior leaders. And in other instances, coaching has been cited as the reason for people leaving organisations to pursue a different career. These are three kinds of impact that use different measures of value and ones that are not easily comparable. For example, is a 10 per cent increase in sales worth more or less than a 10 per cent increase in confidence?
Furthermore, in each case we would need to decide who is an appropriate arbiter of value. It might be the coachee, their boss, their peers, the customer, stakeholders, the company accountant and so on. Each of which might have a legitimate partial, if not full, claim, to be an arbiter, and each would have a different ability to see the causal connections between input (the coaching intervention) and the impact that they see arising.
These may sound like problems and not solutions to the ROI question but so many organisations clumsily make claims of impact without having properly taken them into account. For what its worth, my view is that the sense-making being at the centre of all of this – the coachee – has to be the key arbiter. Others have a role too but the best gauge of how much value has been delivered will always be the coachee. And this will be ascertained by the skilled application of qualitative assessment techniques and not the
pseudo-science of quantitative or even monetary-based assessment processes.
Targeting coaching intelligently
A more helpful perspective in this might be how to extract the most value from coaching. In addition to what we have discussed already under the headings of required inputs, outputs and stretch factors, is the merit of intelligently targeting who should receive coaching, which
might include:
- Those in senior positions (who cast a ‘long shadow’)
- Those with an aptitude for learning
- Those with the potential for senior roles
- Those engaged in innovation
- Those with ambitious goals to achieve
- Those who have undergone change, but are still in a process of transition.
Celebrate the messiness
Coaching engages with the messiness of people and workplaces and offers insights and strategies that are equally various. It works on the sense-making processes that drive the real and lasting transformation within organisations and these effects are not always immediately visible to the onlooker. You run the great risk of missing the point when you apply a physical science lens that seeks to standardise the process of coaching and discount the multiplicity of long- and short-term outcomes that it creates.
People are diverse, flexible, creative, sometimes opaque, beings filled with potential. It just so happens that this is exactly what is needed to create the agile, resilient and innovative organisations that will meet the challenges of the future and flourish. Let’s set up coaching in the best way so that it can continue to support and develop these essential qualities and allow it to finally step out of the dock.
