Measuring ROI is a fool's errand when it comes to learning and development. It's time to stop chasing this false grail and focus on the things that matter.
Organisations need to stop pursuing the myth that we can (or even should) calculate the ROI of L&D efforts, notably for essential skills training, and especially leadership development. Leaders must stop hiding behind claims they need ROI data to decide if developing leadership or other essential skills is the right thing to do for their organisations.
The investment in L&D salaries, education, and work is better focused on gaining a deep and evolving understanding of the business drivers of our organisations and then ensuring our L&D work aligns directly, always, and only with business needs. (Keeping in mind some business needs will be necessary and immediate, and others are just as necessary but serve a longer view.)
A very simple example
We calculate the ROI of a keyboarding class for administrative assistants in the following way: prior to a class Sue takes a keyboarding test and types 40 words per minute. After class she takes a test and now types 60 words per minute. We can calculate how many more documents she produces in a day and figure the ROI for the class.
We could also test her at intervals post-training to prove our hypothesis that she will continue to type faster as she practices. Assumptions critical to this example: Sue was in the correct class given her job, the skill she learned is something she uses most days, and for which she receives immediate feedback.
When the ability to calculate ROI falls apart
Here’s the rub: the ability to calculate ROI falls apart as soon as we move beyond any basic technical skill.
Let’s take delegation, an essential but basic supervisory skill which includes a supervisor knowing which tasks can be given to others, who’s ready for which task, how to answer questions and give effective feedback.
Now add to the mix the supervisor being able to manage the work while not doing it themselves and still retaining accountability for it being done properly, and who is actually willing to give up particular tasks. The ability to delegate effectively means the supervisor is free to do more complex work. The team can produce more work because people typically expend more discretionary effort on work they are good at.
The team will also have new learning opportunities which fuel both engagement and a supervisor succession pipeline, both of which keep organisational costs down by building capacity and aiding retention.
Now help me find the ROI of a delegation skills course that could be objectively quantified in a way that would satisfy a numbers-driven CFO.
It gets harder as we move up the food chain
Moving up the skills-food-chain to 'leadership skills', things get even fuzzier.
First, we have to assume one can actually teach leadership (a roughly $12 billion dollar a year industry).
If ROI is your primary performance measure you should probably find another job
We expose leadership development participants to models, experiences, and readings. We engage them in thinking about business issues, and put them through realistic simulations. We self-assess, peer-assess, and 360 them, including customer feedback and actual business results. (Interestingly these also include HR metrics for which managers are accountable – and which directly impact expenses – such as turnover, absenteeism, stress claims).
At what point do we deem our participants leaders or merely survivors?
Let’s say post-programme one of our new leaders receives an improved rating on their leadership abilities. Was it the class? Prove it. What does that actually mean to the business in dollars? Prove it.
The only question is whether developing leadership capabilities is the right thing to do for your business or not. If you think about it, there’s nothing more perfect than the ROI of actual, effective leadership since it is vital to achieving business results and it is free to do; you just need to know what it is and then do it, right? (We’re now back to where L&D comes in.)
The ability to calculate ROI falls apart as soon as we move beyond any basic technical skill.
What do we do as an alternative to ROI numbers?
We need to take a longer, more strategic view of 'data' – quantitative and qualitative information that’s collected with discipline over time – a portfolio of work that is a true reflection of what’s in L&D’s control, sphere of influence, and results over time.
It's time to use a no-budget approach. This is not a no-accountability approach, but don’t waste valuable time and effort producing numbers and data on things that aren’t really or directly quantifiable to everyone’s satisfaction and then use that shaky foundation to decide whether to spend money or not. (L&D professionals, if you need a good cry, think about how much time you’ve spent trying to prove ROI and what you could have been doing for the business instead.)
Moving up the skills-food-chain to leadership skills, things get even fuzzier.
For L&D leaders and teams, it's essential to collaborate with business leaders on desired business behaviors well before beginning any design.
Be explicit about what’s reasonable. Don’t over-commit, but rather work every moment to ensure your business clients experience the impact of your work to the point that its substantive value is unquestioned.
To all organisational leaders, I'd advise that you hire really well, equip and educate in an ongoing way, and then step back and trust the professionals you hired.
Why did we ever start trying to calculate ROI?
It started off well intended: to see if we’d made progress, to see if X was worth doing again, and to make choices between alternatives given limited resources.
We should not chase ROI to decide whether something is right for the business, nor to prove the L&D team deserves to keep their jobs. (Frankly, if ROI is your primary performance measure you should probably find another job.)
My own measures of L&D impact include the following:
- Growing the number of new and repeat customers
- Being engaged by the business earlier in the process
- Working on more complex projects
- Expanding the nature of the work (moving from just holding classes to consulting and coaching).
Leverage your growing credibility to try new approaches. Don’t get me wrong, go ahead and solicit student and supervisor feedback, examples of shortened time to proficiency, stories of noticeable changes in behaviour, as reported by the business.
Don’t forget to grow the L&D team’s capacity year over year. That’s your L&D portfolio.
L&D: If you need a good cry think about how much time you’ve spent trying to prove ROI
To build on what Dr. Robert Brinkerhoff teaches, if you are billing L&D as a 'business partner' all year, I think it’s disingenuous to stand, hat in hand, and ask to carve out L&D’s share of the credit for business results at year’s end.
A more worthy goal is to work to be indispensable to your business customer, who can then say they wouldn’t have made their goals this year had it not been for the help of L&D.
Stop running after ROI
Given decades of exhortations and worry, and given all of the intelligent people in the L&D community wouldn’t we have found a way to do this more easily by now?
ROI is a false grail – and echoing the words of the character Dr. Henry Jones Sr. in the movie Indiana Jones and the Last Crusade, a man who’d spent his life searching for the Holy Grail only to find in the end that it wasn’t worth it, we need to “let it go.”
Interested in this topic? Read Eight way to maximise your training spend.
13 Responses
I agree completely –
I agree completely – especially stating that ROI calculation depends on the level of learning. On http://www.simulations.co.uk/Learning%20Investment.htm I suggest three levels – operational, tactical and strategic that equate to capital investment levels. A focus on ROI for capital investment means that the company only invests in operational assets and does not invest in strategic change (did Google calculate ROI before building their search engine – I doubt it). Likewise, a focus on learning ROI is at the expense of strategic learning.
There are a lot of things we
There are a lot of things we do – correctly – in L&D which are difficult to quantify monetarily, and although the chain of cause and effect may (and should) lead to a business benefit the degree of causation may be subject to many “contaminating factors”. It is often more important to do an honest justification up front of the balance of costs and benefits (financial and non financial), but most important is to set measurable objectives for the change that is anticipated as a result of the intervention – whether at individual, team or organisational level. This is so often lacking – and it would define the level at which evaluation should be done.
I totally disagree about L&D
I totally disagree about L&D being a false grail. The challenge is investing the time to fine the right outcomes to be measured and as you noted to align those outcomes to the desired goals. As you move up the food chain, other outcomes can be included such as employee retention; external and internal satisfaction to even profitability.
Hi Monica, I agree
Hi Monica, I agree
I think that trying to find ROI for L&D solutions is based on a faulty assumption. Learning is one solution and not the problem or the goal and L&D on its own is never totally successful. I think if we take a performance gap approach and measure what happens to the gap (usually KPIs) then L&D is usually a part solution and we are partners with business in closing the gap. We don’t claim that the performance improvement was all down to L&D but we are one of a set of solutions. When we take a performance consulting approach we can align L&D with the business and its outcomes. Hope this make sense. Nigel Harrison
Love your provocation, if
Love your provocation, if that is what you intended. The fact is I agree ROI should never be a holy grail. Just like the pursuit of profit is a consequence of running a successful business not the purpose.
The fact is that the journey you embarked upon is just good practice. The real issue is being able to ensure that learning has impact and performance improvement/behavioural change is key to ensuring that L&D adds value.
At LTSGlobal (www.ltsglobal.com) we work with the 16 enablers to learning transfer to ensure that the investment in L&D has impact. Based on 15 years research by Prof. Ed Holton, and validated in 24 countries we make sure that HR has the data (not just anecdotal feedback) to ensure that learning has impact. Once behavioural change is aligned, the 16 enablers are switched on and the impact of the the learning becomes explicit. It also aligns all the stakeholders in the learning process so everyone is clear what their responsibility to a behavioural change is and work on improving it. Thanks for adding to the debate. The problem is when times get tough the CEO either sees L&D as not needing justification or the L&D are the first out of the gate when times get tough. We do need to be able to provide evidence of our impact and added value
I really agree. I am
I really agree. I am currently seeking work after an illness but so many times my line manager would be a commercial one and the focus on ROi numbers is palpable. It’s a breath of fresh air when I’m interviewed or employed by someone who understands L&D
I completely agree with this.
I completely agree with this.
Not only is calculating ROI for learning difficult, it’s also a waste of time in most cases. In the past, I have fallen over myself to demonstrate ROI to business leaders who weren’t in the least bit interested. Once their project got delivered, they didn’t want to spend time on the ROI, they were looking to the next thing. Most of the time, the business isn’t measuring the impact of projects in ROI terms. Should L&D be capturing more MI than the business? If the business doesn’t measure it I’m not the L&D team can.
It seems that ROI often matters more to L&D teams trying to justify their existence than it does to the business. If you help business leaders achieve their goals, no justification is need-you are valued!
ROI is great if it’s appropriate but I think return on expectation is more important.
Side note
Side note
I once worked in an L&D team who collaborated with a prominent business school to create a set of standards and certification for L&D capability. I had a lengthy disagreement with an academic who was of the opinion that you must always be able to show ROI. While she was correct in theory, my point was this:
What is the measure of success and who decides? In business, a customer buys (even internally within the business) a learning intervention. They do this with a purpose in mind. They, as the customer, are the ultimate arbiters of whether you delivered or not. If they are happy that you have delivered what they wanted, you did a great job. If they perceive that you did not deliver, you have a problem.
The academic was totally stumped.
an academic totally stumped..
an academic totally stumped….that must be a first. Did you assess causality? 🙂
an academic totally stumped..
an academic totally stumped….that must be a first. Did you assess causality? 🙂
I had a disagreement about
I had a disagreement about this when completing the old CIPD certificate in training practice, (I’m talking 1990’s here). Their view was that you must always show ROI whereas my project wouldn’t show ROI for at least 3 years which was a strategic decision made by the organisation. They would have none of it and so I didn’t complete the programme.
I think you can and should
I think you can and should seek RoI in whatever form/formula. Just the thinking and doing is worthwhile. I also believe you are conflating a number of factors too in your article. One assumes that when L&D gets involved and lets say leadership development as you think it is fuzzy…(.I am not sure it is). One would assume there is a reason to make the investment in the first place. Lets say the metric being observed is poor delegation(your example). One would then look to assess root cause and what would be the fix. In doing so, one can construct a quant metric to this and have it verified by a CFO/FD. The only question then is if the metric moved in the right direction was it as a result of the training. I would then say given the proper pre analysis etc etc before training that the argument about did the training alone cause the improvement is like fairies dancing on the tip of a pin. Interesting at meta physical level but denies the more important aspect…further analysis/inspection of cause/effect. Hence why we use with rigour Kirkpatricks measurement scorecard approach. So I would say start running for RoI. My simplistic view on the matter Cheers.
The baby is flying out the
The baby is flying out the door with the bath water. The drive for training ROI is rooted in the need to show value for the L&D investment. Proving value is an essential and attainable goal for any L&D professional. It’s the ROI metrics that are flawed. Connecting training to results without identifying what people do differently because of training is bad research and leads to erroneous conclusions. The golden ticket to documenting value is to determine what people do differently as a result of training. Answer the do differently question, and you can show value.