Sue Harley, Managing Director of IQdos, looks at the whys and wherefores of evaluating the return on your e-learning investment.
e-learning is a growth industry. IDC predicted in February this year that the worldwide e-learning market will grow almost fourfold in just four years, from US $6.6 billion in 2002 to $23.7 billion in 2006. The European e-learning market will be worth nearly US $6bn by 2005, accounting for 25% of Europe-wide IT training spend.
Why Evaluate?
It seems certain that the e-learning market will continue to grow. More players will enter the market, competition will intensify, as always with industries in this early stage of the product life cycle. The Financial Times noted recently the attempts of generalist HR software and ERP platform vendors to 'muscle in' on the e-learning market. More and more, e-learning providers will have to prove themselves in what will become an ever tougher market.
One way to do this is to demonstrate the business benefits their products offer. This will help e-learning vendors in several ways. Firstly, it will help them to stand out in a market set to become crowded. Also, if purchasers and users of e-learning can justify investment in e-learning to their bosses, or suppliers can do so to their clients, then surely purchasers will be prompted to buy more.
Lastly, and more fundamentally, surely all companies must analyse the ROI on any purchase, whatever it may be. e-learning will be subjected to the same scrutiny as all investments, and needs to stand the test if the market is to live up to its predicted potential.
Get Over It!
The common perception is that the impact of e-learning is 'difficult' to evaluate - this is simply not so. E-learning naturally lends itself to detailed evaluation at both trainee and company-wide levels. The ways to achieve this are many and varied, and different approaches will suit different organisations.
When it comes to monitoring the individual trainee, the very nature of e-learning is that it is delivered through technology- what better method to capture and store data on actual e-learning use? Also, e-learning solutions often feature a Learning Management System (LMS), or in-built learning modules with tracking capabilities. With such automated tools at the e-learning user's fingertips, evaluation is there on a plate!
Most firms have structured HR policies in place to monitor and appraise staff performance closely, so it is easy to register whether or not improvements are made following e-learning courses. Again, this can be done online, using tools that some e-learning suppliers offer.
At the enterprise level, companies closely monitor their overall performance. So improvement levels following training in, say, sales techniques are more easily quantifiable. When it comes to return on money spent, what company doesn't keep a close eye on its expenditure, be it on training or anything else.
Some e-learning providers actually offer tools to measure company-wide training levels and skills gaps very precisely, making the evaluation of any investment in e-learning all the more accurate and thorough.
How to evaluate e-learning
So how do we measure the effectiveness of e-learning? Whilst there is no magic formula, there are several ways to achieve this over the short, medium and long term. It is a case of demonstrating the positive impacts that courses have on business performance.
In the immediate term, firms can monitor how employees feel about the training they receive, via basic feedback and evaluation forms ('happy sheets') - this can be carried out online for faster and more accurate data capture and analysis, and should ideally be done before and after the course.
Fundamentally, it is a case of finding out what has been learnt, what new skills have been gained, and how much has been retained. This may sound obvious, but measuring this is crucial - improved skills and knowledge are the very reason why we train. And yet, it is an area often neglected, with those responsible for training not seeing course evaluation through.
Once staff have gained new capabilities, companies expect to see improvements in their performance over the medium term. Are recent trainees performing new tasks, or better at old ones? Are they more confident, more competent and hopefully happier in their roles? In essence, are they achieving better results?
Again, this should be easily observed, be it on a day-to-day basis, in regular performance appraisals, or in improved measurable results (e.g. improved sales, less problems, etc.).
Comparisons with performances prior to training, or with staff who have not had the same training, can also be helpful, all of this does take dedication and staff trained in coaching.
Ultimately of course, organisations investing in e-learning want to see an impact on the business as a whole. Have revenues increased since a sales training course was brought in? Have complaints or faulty products decreased? Is productivity up, are cost savings evident?
When evaluating effects at the macro level, companies must taker a longer-term view, as it may take several months or more to train a whole department and see results come through.
As we have seen, there are e-learning and e-performance tools to help here - web-based solutions which provide accurate 'snapshots' of companies' skills levels and gaps at a given point in time. It is worth reminding ourselves that e-learning itself can be part of the solution when it comes to demonstrating ROI on e-learning!
As noted, there is no set-in-stone recipe for measuring how e-learning has benefited an organisation, but a 'menu' of ways to look at this, from which companies can choose which work best for them. However, one IQdos client represents a great example of a structured approach to evaluating the ROI on the e-learning they buy.
The company tends to spend around £10-20,000 on the design and production of each complete, high quality e-learning programme, and works to a benchmark of approximately 9 months to see a full return on their initial investment for each project.
To ensure this is achieved, the firm has created its own ROI model, which sets out all costs involved, including travel, accommodation, opportunity costs, trainer time, etc. Each e-learning programme is audited fully according to this model before commencing.
A Natural Success Story
We have seen how e-learning is naturally given to evaluation, and there is an equally good case to argue that it offers a clear advantage when it comes to demonstrating actual return on investment (ROI) - i.e. the relatively low cost of solutions. In a nutshell, e-learning can offer benefits in staff performance for less overall investment.
Computer Weekly reported in March that Ford's 'Learning Network' - which integrates all training, online testing and learning resources for all 300,000 employees worldwide - is set to save the car manufacturer a massive $17m. A 2001 report from e-Ventures.com saw companies reporting up to 75% reductions over 5 years in training costs by adopting e-learning programmes - a figure submitted by the likes of Sun Mircosytems and Bayer, no less! Others quoted cost savings in the region of $50m.
ROI is a simple factor of benefits over costs. e-learning is a low-cost way to implement high quality training. The lower the costs, the better the ROI ratio becomes.
Time to act
As we have seen, the ability to evaluate the impact of e-learning, and demonstrate its benefits, is essential to the success of e-learning as with all products, and will become more and more crucial as the market becomes more competitive. Though exact evaluation will always be an imprecise science, it is not difficult to carry out, and there are demonstrable ROI benefits to e-learning.
Yet, the industry is not yet offering this as a given. The e-Ventures.com report concluded that "e-learning assessment is still at an early stage". Few firms were offering evaluation tools at the time, according to the report - this despite the existence of such solutions and the imperative to prove ROI.
Thus, the next step for all of us in e-learning industry will be to use the capabilities of the technology we offer, to show firms the returns they want to see. It is the companies who achieve this which will continue to flourish as the market hots up.
