Why do we evaluate professional development?
Main points:
What is evaluation?
What should we be getting out of the evaluation of professional development?
Who are the big names in evaluation?
If you're new to the concept of evaluation then read on. The Open University's Viv Nunn gives us an introduction and talks us through the benefits.
What is evaluation?
"Evaluation is the comparison of actual project impacts against agreed strategic plans."
(Janet Shapiro 'Monitoring and Evaluation', CIVICUS)
Evaluation is a journey – it starts at the beginning with your organisations plans. These plans inform your training needs, and provide the metrics against which you can evaluate success. From increasing efficiency and productivity to addressing skills shortages and retention, evaluation results tell you what actually happened, and how well you are achieving your goals. It identifies what is helping you get the most from your training, and indicates how you might maximise these benefits. And it points the way forward – what next?
What should we be getting out of the evaluation of professional development?
- Evidence of the extent to which the professional development is contributing to your organisation's success
In the current economic crisis, it is more important than ever to demonstrate conclusively how professional development is contributing to the success of your organisation.
Evaluation results therefore need to link:
- The actual learning achieved
- Real behaviour change
- The resulting organisation impact
Results also need to be in a language suitable for its audience – maybe a summary capturing key success indicators for senior management, and a more qualitative report for others in the L & D function who might want to explore raw data, results and recommendations more fully.
- Validation that the correct learning solution has been identified and suggestions for programme improvement
You need to know if the learning solution you have chosen fits the bill – is it delivering the knowledge that is needed in the best possible way for your learners? Is the content relevant and timely, the commitment manageable, and the level appropriate? If not, what improvements could be made to the pedagogy, design or delivery which would enhance the learner's experience, and increase the transfer of learning back into the workplace?
"If the workplace learning environment isn't right...there will be little or no impact to report, regardless of how effective the actual learning solution has been."- Advice about how to get the most from your L&D budget by considering the workplace learning environment
Evaluation results should give you feedback about what is helping (and hindering) the transfer of learning back in the workplace, and make recommendations about what you could do to increase this. Research shows that good preparation, appropriate workplace learning support, encouraging learners to use their new skills and knowledge back at work, goal setting and monitoring progress, and the reinforcement of key themes both during and beyond the end of the programme all influence this vital part of the learning process. Considering your evaluation findings and implementing recommendations will help you get the most from the benefits of your investment in professional development.
Who are the big names in evaluation?
The most prominent leader in the field of evaluation is Donald Kirkpatrick, whose '4 Level Model' grew from his PhD thesis in the 50s and 60s. More recently, son Jim has further established his work, re-emphasising the need to evidence the links between learning outcomes, behaviour change, and business needs [1]. This model fundamentally underpins several other evaluation theories (including the first four levels of the Jack Phillips ROI model) but it is often denigrated because of bad press around 'happy sheets'.
Good evaluation can give you evidence of much more than just the link from learning to your organisation's success. While Jack Philips isolates the effects of the learning solution from its context in his ROI calculations, Josh Bersin really emphasises the role of the learning environment, suggesting that up to 70% of learning takes place back on the job after the formal learning has ended. Establishing what helped transfer learning back to the workplace (such as line manager buy-in and support, opportunities to apply learning back at work etc.) can provide valuable insights to help you maximise the return from your educational spend for future cohorts.
If the workplace learning environment isn't right, if learners cannot find opportunities to apply what they have learned at work, if the organisational culture is not one which encourages them to try out their new skills and knowledge, there will be little or no impact to report, regardless of how effective the actual learning solution has been. Isolating learning from its context will not provide useful data alone. Evaluating the context of learning along with the learning solution will give you the greatest opportunity to get the most from your investment.
Conclusion
Evaluation is a journey – it compares the actual impacts of learning against your agreed strategic plans, be it development of skills to deliver competitive advantage or up-skilling staff to improve performance. It provides evidence of the extent to which the professional development is contributing to your organisation's success; it provides validation that the correct learning solution has been identified and suggestions for programme improvement; and it provides advice about how to get the most from your L & D budget by considering the workplace learning environment.
Viv Nunn is the learning evaluation and development services manager at The Open University. You can find out more about her role by clicking here
[1] Kirkpatrick, J. and Kayser Kirkpatrick, W. 'The Kirkpatrick Four Levels: A Fresh Look after 50 Years'
