Learning and development: why measuring ROI is a false grailby
Measuring ROI is a fool's errand when it comes to learning and development. It's time to stop chasing this false grail and focus on the things that matter.
Organisations need to stop pursuing the myth that we can (or even should) calculate the ROI of L&D efforts, notably for essential skills training, and especially leadership development. Leaders must stop hiding behind claims they need ROI data to decide if developing leadership or other essential skills is the right thing to do for their organisations.
The investment in L&D salaries, education, and work is better focused on gaining a deep and evolving understanding of the business drivers of our organisations and then ensuring our L&D work aligns directly, always, and only with business needs. (Keeping in mind some business needs will be necessary and immediate, and others are just as necessary but serve a longer view.)
A very simple example
We calculate the ROI of a keyboarding class for administrative assistants in the following way: prior to a class Sue takes a keyboarding test and types 40 words per minute. After class she takes a test and now types 60 words per minute. We can calculate how many more documents she produces in a day and figure the ROI for the class.
We could also test her at intervals post-training to prove our hypothesis that she will continue to type faster as she practices. Assumptions critical to this example: Sue was in the correct class given her job, the skill she learned is something she uses most days, and for which she receives immediate feedback.
When the ability to calculate ROI falls apart
Here’s the rub: the ability to calculate ROI falls apart as soon as we move beyond any basic technical skill.
Let’s take delegation, an essential but basic supervisory skill which includes a supervisor knowing which tasks can be given to others, who’s ready for which task, how to answer questions and give effective feedback.
Now add to the mix the supervisor being able to manage the work while not doing it themselves and still retaining accountability for it being done properly, and who is actually willing to give up particular tasks. The ability to delegate effectively means the supervisor is free to do more complex work. The team can produce more work because people typically expend more discretionary effort on work they are good at.
The team will also have new learning opportunities which fuel both engagement and a supervisor succession pipeline, both of which keep organisational costs down by building capacity and aiding retention.
Now help me find the ROI of a delegation skills course that could be objectively quantified in a way that would satisfy a numbers-driven CFO.
It gets harder as we move up the food chain
Moving up the skills-food-chain to 'leadership skills', things get even fuzzier.
First, we have to assume one can actually teach leadership (a roughly $12 billion dollar a year industry).
If ROI is your primary performance measure you should probably find another job
We expose leadership development participants to models, experiences, and readings. We engage them in thinking about business issues, and put them through realistic simulations. We self-assess, peer-assess, and 360 them, including customer feedback and actual business results. (Interestingly these also include HR metrics for which managers are accountable – and which directly impact expenses – such as turnover, absenteeism, stress claims).
At what point do we deem our participants leaders or merely survivors?
Let’s say post-programme one of our new leaders receives an improved rating on their leadership abilities. Was it the class? Prove it. What does that actually mean to the business in dollars? Prove it.
The only question is whether developing leadership capabilities is the right thing to do for your business or not. If you think about it, there’s nothing more perfect than the ROI of actual, effective leadership since it is vital to achieving business results and it is free to do; you just need to know what it is and then do it, right? (We’re now back to where L&D comes in.)
The ability to calculate ROI falls apart as soon as we move beyond any basic technical skill.
What do we do as an alternative to ROI numbers?
We need to take a longer, more strategic view of 'data' – quantitative and qualitative information that’s collected with discipline over time – a portfolio of work that is a true reflection of what’s in L&D’s control, sphere of influence, and results over time.
It's time to use a no-budget approach. This is not a no-accountability approach, but don’t waste valuable time and effort producing numbers and data on things that aren’t really or directly quantifiable to everyone’s satisfaction and then use that shaky foundation to decide whether to spend money or not. (L&D professionals, if you need a good cry, think about how much time you’ve spent trying to prove ROI and what you could have been doing for the business instead.)
Moving up the skills-food-chain to leadership skills, things get even fuzzier.
For L&D leaders and teams, it's essential to collaborate with business leaders on desired business behaviors well before beginning any design.
Be explicit about what’s reasonable. Don’t over-commit, but rather work every moment to ensure your business clients experience the impact of your work to the point that its substantive value is unquestioned.
To all organisational leaders, I'd advise that you hire really well, equip and educate in an ongoing way, and then step back and trust the professionals you hired.
Why did we ever start trying to calculate ROI?
It started off well intended: to see if we’d made progress, to see if X was worth doing again, and to make choices between alternatives given limited resources.
We should not chase ROI to decide whether something is right for the business, nor to prove the L&D team deserves to keep their jobs. (Frankly, if ROI is your primary performance measure you should probably find another job.)
My own measures of L&D impact include the following:
- Growing the number of new and repeat customers
- Being engaged by the business earlier in the process
- Working on more complex projects
- Expanding the nature of the work (moving from just holding classes to consulting and coaching).
Leverage your growing credibility to try new approaches. Don’t get me wrong, go ahead and solicit student and supervisor feedback, examples of shortened time to proficiency, stories of noticeable changes in behaviour, as reported by the business.
Don’t forget to grow the L&D team’s capacity year over year. That’s your L&D portfolio.
L&D: If you need a good cry think about how much time you’ve spent trying to prove ROI
To build on what Dr. Robert Brinkerhoff teaches, if you are billing L&D as a 'business partner' all year, I think it’s disingenuous to stand, hat in hand, and ask to carve out L&D’s share of the credit for business results at year’s end.
A more worthy goal is to work to be indispensable to your business customer, who can then say they wouldn’t have made their goals this year had it not been for the help of L&D.
Stop running after ROI
Given decades of exhortations and worry, and given all of the intelligent people in the L&D community wouldn’t we have found a way to do this more easily by now?
ROI is a false grail – and echoing the words of the character Dr. Henry Jones Sr. in the movie Indiana Jones and the Last Crusade, a man who’d spent his life searching for the Holy Grail only to find in the end that it wasn’t worth it, we need to “let it go.”
Interested in this topic? Read Eight way to maximise your training spend.
Known as a highly effective leader in her own right for over 25 years, Monica Goodale holds an Ed.D. specializing in Organizational Leadership from Pepperdine University. Her professional career spans both military and civilian experiences, and she regularly speaks and teaches at professional conferences. For the last several years she has been...