Evaluation: Digital technology applications
What passes for a joke in evaluation circles is that the return on investment (ROI) of calculating ROI isn't worth it. This reflects the widespread assumption that ROI, indeed every form of evaluation, is expensive. It can be so, but the generalisation that evaluation is expensive is no more accurate than comparable generalisations that training is expensive, or marketing is expensive, or any business activity is expensive. Whether it's too expensive depends on how it's done, and the other side of the coin is that not doing evaluation can be much more expensive, sometimes fatally so.
The advent of digital technology that is widely available, easy to use and readily adopted has eliminated much of the expensive/affordable debate in evaluation. However, digital technology can contribute much more. There are five distinct phases of learning evaluation that benefit significantly from the lever of digital technology, and these five phases form a cycle: evaluation planning leads to data collection, then storage and analysis, to reporting and from there to planning again.
The important thing to remember is that this is not just about efficiency, but the effectiveness of learning evaluation. Evaluation planning is the stage when organisations can devise strategies, accompanied by scorecards and dashboards to measure and manipulate them, and this is where digital technology comes into its own. This is a powerful way to secure buy-in, to brand the evaluation, and to tie it into other systems in the organisation.
Examples of data collection activities enhanced by digital technology include: distribution of surveys; audio and video recording; virtual meetings.
Examples of data storage advantages from digital technology include: reduced storage space; greater security; different formats held in the same place.
Examples of data analysis benefits from digital technology include: faster calculations; easier identification of themes and trends.
Examples of reporting benefits from digital technology include: faster report generation and distribution; better quantitative and qualitative reporting; multi-media capability.
"As a general guideline, buyers should opt for the bespoke in preference to the generic" Summary
There are some good off-the-shelf digital tools available, but these are rarely integrated to address all of the five cycle phases described in this paper. And where they are, they usually constrain the user within the limits of their templates. A better option is to use the most generic software tools, such as Microsoft Access and Excel, or SurveyMonkey – the same everyday tools used for other business applications – and integrate these into corporate systems everyone in the organisation is already comfortable using. This may still mean some bespoke development, but this should then be affordable and lower risk.
Deployed in this way, digital technology does not just leverage more from your learning evaluation, it may add significant value.
You might also be interested in
Charity Director. Consultant in OD, strategic HR and leadership development. Author of five books including Delivering E-Learning (Kogan Page, 2009). Interested in strategy, OD, leadership, technology, evidence and evaluation. Available for writing and speaking.