The 'Bad Science' of training
Training trends come and go. Shiny new methodologies can all-too-quickly appear tarnished once the 'next big thing' is on the scene. Garry Platt investigates why a new approach so often seems like a better approach to learning.
We have read recently on TrainingZone how elearning was effectively dead in the water and as a strategy was defunct for most organisations. We then read a riposte which claimed that classroom-based development was totally inefficient and only lunatics use it.
Shortly after this a series of 'Indignant of Tunbridge Wells' postings appeared following a short piece I wrote suggesting that regardless of what an academic researcher working in Burkina Faso might have to say about a particular theory or approach, personal experience and actual results should also play a part in our adoption and choices of developmental processes.
All this I believe reflects the ebb and flow of trends, fashions and preferences which are a constant factor in the field of training. Methodologies and approaches can be said to have seasons and once over they are spurned by the cognoscenti being viewed as ‘old hat’. That is not to say they do not from time to time re-emerge and come back into vogue but usually only for a short time.
I recently read a fascinating book by Ben Goldacre called Bad Science. It explores just what is bad science, how to spot it and examines claims made by both researchers and non-academics based upon poor data or no data at all. Within the book there are sections dedicated to ‘the internationally acclaimed holistic nutritionist’ Dr Gillian McKeith, homeopathy, fish oils for enhancing learning and the great MMR vaccine debacle, all these and more are referenced. But one particular section gives a fascinating insight into the workings of the human mind and body and I also suspect throws some light onto why methods come and go in our industry and why they might work for some but not for others. Whilst our psychological and physiological responses to medicine are not the same as our experience and interaction with training methods I suspect that many of the same factors are in play.
Let me quote the following section from the book, (page 81):
"Moerman examined 117 studies of ulcer drugs from between 1975 and 1994, and found, astonishingly, that they interact in a way you would never have expected: culturally, rather than pharmacodynamically. Cimetidine was one of the first ulcer drugs on the market, and it is still in use today: in 1975 when it was new, it eradicated 80 per cent of ulcers, on average, in the various different trials. As time passed, however, the success rate of cimetidine deteriorated to just 50 per cent. Most interestingly, this deterioration seems to have occurred particularly after the introduction of ranitidine, a competing and supposedly superior drug, onto the market five years later. So the self-same drug became less effective with time, as new drugs were brought in.
"There are a lot of possible interpretations of this. It’s possible, of course, that it was a function of changing research protocols. But a highly compelling possibility is that the older drugs became less effective after new ones were brought in because of deteriorating medical belief in them. Another study from 2002 looked at seventy-five trials of antidepressants over previous twenty years, and found that the response to placebo has increased significantly in recent years (as has the response to medication), perhaps as our expectations of those drugs have increased.
"Findings like these have important ramifications for our view of the placebo effect, and for all of medicine, since it may be a potent universal force: we must remember, specifically, that the placebo effect – or the ‘meaning effect’ – is culturally specific.”
I was reminded immediately of approaches and methods which as I have alluded to already arrive and then eventually fade: Emotional Intelligence, The Learning Organisation, Left/Right Brain, even coaching is starting to look like that cardigan you bought last year, and will probably now stay in the bottom of the wardrobe until the next mass extinction event.
Our response or anticipation for shiny new things and how this makes us experience ‘old’ systems might well indicate why elearning gets stonewalled by so many trainers and developers. They experienced and remember the first wave of this approach when it promised everything and delivered virtually nothing. And now despite the fact that this methodology has got its act together it is difficult for some people to turn around and go backwards, as they perceive it.
As we have read a medicine whose make up and properties have not changed apparently loses some of its potency when a ‘newer’ drug becomes available. We might speculate that the same is true with training. It’s slightly unpalatable, but a process that you use or adopt might lose its effectiveness and consequently its veracity not because of anything you are doing wrong but simply because the perception of it by either the providers and/or the end users negates some of its effectiveness.
Of course some methods were and are just crap from the outset and entered the training and development communities orbit on the back of misguided promotion and spin, in affect; the b/s du jour, i.e. fire walking, bongo drumming, Mozart Effect, etc.
I can recommend Ben Goldacre’s book. It’s funny, easy to read and incredibly enlightening about understanding research findings, scientific claims and what might and might not be legitimate.
Garry Platt is an experienced training consultant . He has worked with a number of international organisations helping them to enhance their approach to training and development. Examples of some of the organisations he has worked with in 2012 are outlined here; Siemens, Fenman, Formica, Mauritius Employers Federation, Wabtec...