Return on Expectations: The ultimate demonstration of training value

Main points: 

How has the training landscape changed since the Kirkpatrick Four Levels were introduced 50 years ago?
We have to demonstrate the value of our training offerings even more so in today's competitive market
ROE is a positive measure that pulls an organisation together in the quest to define and achieve the target

As the 50th anniversary of The Kirkpatrick evaluation model approaches, Jim Kirkpatrick, PhD and Wendy Kayser Kirkpatrick offer background behind and a preface to the latest Kirkpatrick white paper, The Kirkpatrick Four Levels: A Fresh Look After 50 Years.

 

We have recently unveiled the current, true, comprehensive Kirkpatrick Model that was created 50 years ago by Donald Kirkpatrick. We believe that “Return on Expectations” or ROE is the most meaningful way to demonstrate the value of training (and reinforcement) efforts to the business. Proclaiming ROE as THE measure of training value is a bold statement, considering that the most popular and recognized way to demonstrate value is ROI, or “Return on Investment”.

As you might guess, the “expectations” we are referring to in ROE are those of our internal and external business partners – the people we are in business to support. Jim was addressing an ASTD Chapter in Fort Worth, Texas and a learning professional described his job as 'supporting the needs of corporate business.' We know when we hear words along those lines that the person gets it. We strongly urge you to consider the notion that training follows, serves, and supports the work of our business partners, and training is not an end in and of itself.

We often suggest that we no longer have a free pass when it comes to value. For years, we as training professionals have been seen as having inherent value to our organisations just because we are learning and development professionals. Now, however, we now find ourselves in the position of having to demonstrate our value just like everybody else. And that is where ROE comes in.

We believe so strongly that 'The End' is 'The Beginning' that we made it one of the Five Kirkpatrick Foundational Principles. What we mean by this is that before we put together and deliver a training program based on the request of our business stakeholders, we need to find out what they are expecting to see as a result of the efforts. We want to find out 'what success will look like' in the eyes of our business leaders so we can better focus our training, reinforcement, and evaluation to meet those expectations. Thus, if we are clear about specific success outcomes, we are already well on the path to delivering return on our stakeholders’ expectations.

At times ROE looks just like return on investment. It can, in fact, include a formal ROI calculation if that is what the business stakeholders expect. The difference is that ROE is not limited to ROI. ROE may at times be measured in simpler terms. For example, an agreed-upon increase in sales, or decrease in scrap. Or it could also encompass intangibles like better employee morale and loyalty. The great thing about ROE is that the stakeholder is in control of determining what is important to them, and you as a training professional have the ability to determine with their agreement the best ways to measure the results. ROE is stated in language that is understood by the business stakeholders, and everyone within the organisation that has a stake in the training impact.

As mentioned earlier, the terms of ROE are stated before a project even begins. So ROE is a positive measure that pulls an organisation together in the quest to define and achieve the target. In contrast, ROI tends to be measured after the fact, as a defensive measure where a training professional or organization is attempting to isolate the impact of their efforts to justify their existence. We hope you can see the difference.

ROE is a collaborative agreement that unites an organisation in working towards a common goal. ROI is a summative measurement that separates business units by attempting to isolate the efforts of each. Even though ROE may include measurements that are not as scientific and formulaic as ROI, do not be misled to think that ROE is “soft”.

On the contrary, it is a strong measurement that can include all of the metrics that typically make up ROI (if the stakeholders want to see that type of evidence), and more that ROI is not able to measure. Examples of typical 'expectations' converted into success outcomes are:

  • Revenue from new customers
  • Increased operations efficiencies with decreased training costs
  • Reduced turn-around times
  • Increased retention of top talent

There is certainly a financial component in each of these examples. And there is also a long list of “other” benefits that may be measured numerically or not. Typically it doesn’t make sense to attempt to isolate these measurements because they were accomplished as a team. When an organisation sets their targets collaboratively up front and works towards them in lockstep, the entire team may take credit for the success.

There is a link to our latest white paper below that will outline line each of the Five Kirkpatrick Foundational Principles. We believe that no matter your tenure in the training industry, you will find these ideas elegantly simple yet genuinely challenging. We will close with a story that illustrates the importance of principle #1.

The End is The Beginning

I am currently working with the Abu Dhabi Police force, part of the United Arab Emirates. On a recent trip, my connecting flight was delayed and I had to divert from my final destination of Abu Dhabi to Dubai. No problem; just an hour-long cab ride. When I climbed into my airport cab, I asked the driver: “Do you know where the Emirates Plaza Hotel in Abu Dhabi is?” “Oh, surely”, he said, and off we went. When we entered the city of Abu Dhabi, he asked me, “Do you know where the hotel is?” No, I thought, you said you did.” “No”, I said, “I only knew where the city is.” Silence.

Then, as we were riding, my driver honked and flashed his lights at a local taxi until the guy pulled over. My driver got out and entered into a discussion in Arabic with the other driver. He got back in, and away we went. For the next hour, he pulled over at least six more taxis, each time getting out and discussing and pointing with the driver. Finally, we got to my hotel. Metaphors are constantly hitting me in the face, and this terror taxi trip was no exception. It showed me what happens when someone is unfamiliar with the desired end.

It is the same in training. Unless we clearly identify and negotiate what the indicators of success will be in the eyes of our business partners, we will be guessing and hoping, just like my cab driver. We encourage you to not only work with the notion of ROE, but the entire Kirkpatrick Model presented in the whitepaper.

We believe you, like many others, will create a strong, working bond with your business partners that will lead to a higher degree of mutual respect and positive, measurable bottom line results.

Download their new whitepaper, The Kirkpatrick Four Levels: A Fresh Look After 50 Years

Read our recent interview with Donald Kirkpatrick here.

Wendy Kayser Kirkpatrick is the director of Kirkpatrick Partners, LLC. She is a certified instructional designer. Wendy conducts seminars and facilitates corporate meetings. She draws on 16 years of experience in the business world to make her training relevant and impactful with measurable results. Wendy and Jim Kirkpatrick have written a new book entitled Training On Trial, to be published by AMACOM books in late 2009. Jim Kirkpatrick Ph.D. is SMR USA's vice president of global training and consulting, and presents workshops for and provides consulting to Fortune 500 companies around the world. Jim has co-written three books with his father, and has spoke at conferences in four continents just in the past five years.

Comments

Teryt's picture

And it strongly reminds me of Paul Kearns' baseline measurement approach. That is, set the expectation and measurement of outcomes up front.

We have certainly encountered clients who are not interested in specific ROI.  I remember almost having an argument with one, with me mildly insisting that we show solid, monetary ROI.  He believed the real value of the development work wouldn't be shown monetarily.  In the end, we did show ROI as a direct result of his people spending their time more productively, and through some cost saving measures they implemented as well.  This was a nice bonus, but the real value showed up in other ways such as better teamwork, increased morale, etc.  (Unfortunately, not long after this development work their business took a big hit in their specific market, and they actually lost money.  This is where ROI measurement doesn't work well, because it's difficult to show what impact the L&D had on the bottom line - since there are numerous other internal & external forces at work as well.) 

Therefore, we have been using ROE, even before we heard of it from Kirkpatrick.  (Plus we were influenced by Kearns' baseline approach.)

We even guarantee the client will see at least 500% ROI (but I think now we may start using the term ROE).  This is actually easy if you have them set as one of their goals something with a measurable financial impact.

siliconb's picture

Measuring the success of training interventions has always an important consideration for key stakeholders, but its not always that easy to do - espcecially when you're dealing with developing interpersonal skills (for example an Assertiveness & Confidence Building training course).

If you're running a Sales, Business Process Management or Six Sigma course, measuring financial returns is easy because measurement is a key part of the tools themselves - however with interpersonal skills programmes it's much harder to ask the question "if I spend £X on training, what will be my financial return" because the ways the training makes an impact will be evidenced in many non-financial ways, such as employee retention or job satisfaction. Being able to set flexible ROE targets that are agreed with the seems a much better approach in these circumstances.

Colin Welch
Training Manager
Silicon Beach Training

 

schma_m's picture

It seems to me (opinion) that there is a widely held belief that evaluating the impact of say a sales course is easier than evaluating the impact of a soft skills course or putting a monetary value on something like retention.

People attend a sales course. Then sales go up sometime later. So is that down to the sales course, and if so which parts, exactly, of the sales course..? Or is that down to seasonal variations? Or perhaps some unexpected catastrophe that has hit a competitor or led to an immediate rise in demand for what is being sold? Perhaps it's a combination of these factors, and they all contributed to an increase in sales. So how much then did the sales course contribute? And was it worth the disruption of taking sales people off the road, and the cost of having them attend, costs such as travel, accommodation & extra food, as well as the opportunity costs of having somebody NOT sell while on the course?

And a soft skills course - does that reduce the liklihood of somebody leaving? Does retention improve? What is the cost of losing somebody, taking time to hire a replacement and in that time missing out on the benefits of that missing person's skills, knowledge, contacts etc? What is the cost in the end, in hard $$ of the recruitment fees? And then the training and time taken (on-going opportunity costs racking up by the hour...) to get them up to speed so that they are a true replacement for the person who left? Does that soft skills course improve relations and/or communications such that there's less duplication of effort, speedier communications that result in less mistakes and wastage? Seems to me there's plenty here to link explicitly any soft skills intervention to hard $$.

The trick is in the 'how' and it's the same 'how' for delaing with the question of the sales course at the start of my reply as it is in dealing with the soft skills course.

Does it matter what the pure ROI is? Probably not, at least, not often, and not to many. If I spend $500 on training a sales person who brings in $4M a year, is that money well spent? I don't know. Why don't I know? Because I don't know what would have happened if I hadn't spent the money in this way. Could I have given the $500 to the sales person and said "have a good time!" if I was equally sure that in the end I'd still get my $4M in sales revenue? Which one would have been more motivational, more likely to get 'buy-in' from the sales person, so that s/he'd be more likely to get on board with other things I've got planned, such as a more rigorous sales performance management approach (and sales people hate being managed!)?

ROI has it's place, whether the investment is $$, political or social capital, or 'fun points'. And so does ROE or return on Expectations. But can't we just keep it a little bit simpler and clearer?

Can't we just start off by being incredibly clear about what future outcome we want, starting with what is the minimum acceptable, and describe that in terms of what people are thinking, saying, doing and feeling, as well as key result areas like sales value, and key performance indicators like cycle times?

Then we can work backwards, figuring out the tasks, actions and processes etc needed to deliver that future outcome. We bare in mind the values and beliefs we want to live by.

And with those tasks, actions and processes etc identified, we can figure out what skills, knowledge and attitudes are needed to deliver those tasks, actions and processes, in the spirit of our beliefs and values we want to live by.

Now we have the kind of detail by which people can look at themselves as well as each other, to see where any gaps may exist in skills, attitude, knowledge, behaviours, processes, equipment and all the other things we need, and be very clear about what's needed to close that gap. And in doing so, every single action taken, skill and attitude developed, every piece of equipment/facilities modified or bought, is clearly linked to the future outcome we want. And if we value cost effective, we'll buy cost effective. And if we value something a little more extravagant, or we value safety such that we will always have 4-fold redundancy and ensure designs are done by 4 different design houses and built in 4 different factories for example, that we accept the higher costs because that's part of who we are...

And we can be happy in the knowledge that none of it is there as a luxury or nice to have, that all of it is there to ensure we get the minimum desirable future outcome.

I'm not saying it's easy or cheap to do. I am saying it's easier and cheaper to do than so many think. A lot easier and cheaper to do.

I'm curious to understand what has really been the problem with this for the last 50 years? Seriously. I can speculate, and I have some views. But really, why didn't we get it as a profession all these years, and why don't we seem to be getting it now? Is it simply a matter of belief?

Yuvarajah's picture

Thank you Kirkpatrick, for being bold enough to finally say what is really "meaningful" in measuring training. 

For years on, I had struggled with having to argue that not all training undertaking MUST be translated into ROI, in monetary terms. It gave the impression that it was the only way to convince the finance in bidding for the training budget. I am elated to hear the call for ROE as making more sense and meaning because it dilutes the obsession over "co-relationary" measures. Now, training professionals can get down to lending more credit and trust to the learning discipline. They can free-up their "distractions" and give greater attention to reaping the desired outcome of training investment.  

If the end is the begining, then I would say "the begining is the answer". By this, I mean doing a proper TNA. Without it, you will not be able to address or measure 'ROE". How are objectives ascertained?. You must be clear upfront with the end state before you can identify the objectives. Honestly speaking, how many design their training to the depth and level of incorporating into their "terminal" objectives, performance KRA and KPIs. Do the beneficiaries of the training effort clearly communicate the details of the pre-training gaps and post training expected outcomes to trainees?. Most importantly, do both managers and trainees treat training as a mission critical success factor for work place performance improvement?. 

Training is not a panacea, but learning is. I hope we don't have to wait another 50 years to realise that.    

 

Back to top Back to top
wanting-removable