A news and idea service of Bill Communications (Lakewood)
Tuesday, Sept. 28, 1999 Vol. 2, No. 27
*************************************************************
Anyone. Anytime. Anywhere. http://www.click2learn.com
*************************************************************
THIS WEEK:
1. Adapting Kirkpatrick
2. Who is Trainer 2000?
3. More about re-using games
4. Who's going to the shows?
5. Window shopping: Free demos, sales horror stories
_____________
ADAPTING KIRKPATRICK ONLINE
Gauging the effectiveness of training online means applying Kirkpatrick with a twist, says Saul Carliner ([email protected]), a training consultant who teaches information design at Bentley College in Waltham, MA.
Much online learning -- especially learning via performance support and knowledge management -- is "informal learning," Carliner says. Measuring it goes beyond the traditional four- level Kirkpatrick model for evaluating training's effectiveness.
Indeed, the Kirkpatrick model may not work with online learning at all, cautions Carliner, because online learning doesn't always have firm learning objectives.
"We can measure what was learned, says Carliner, a presenter at OnLine Learning '99 in Los Angeles next month. "But that might not be convincing to managers."
What will convince managers? Find out what management values, suggests Carliner. Then gather evidence that shows your training supports those values. For example:
o Does the online content make the most effective use of
the organization's resources? "To demonstrate this
requires good business cases for each possible scenario,"
says Carliner. Follow through to show that the learning
delivered intended results.
o Were participants and managers satisfied with the
training? Collecting smiley-face evaluations from
online courses can be difficult.
o Did participants learn the material? Testing and
certification will tell you.
o Do participants use the material on the job? Survey
and observe to find out.
o Did the training improve business performance? Write a
business objective for the course in addition to content
objectives. Then use standard business measures to see if
they show better performance: Did sales meet targets? Did
customers call the help line less?
o Did learners like working with the trainer? This sounds
like a popularity contest and, unfortunately, it is to
some extent. A bad trainer may diminish learning.
Evaluating online learning "should be a way of business,"
concludes Carliner, "just as it has become a way of business
for classroom learning."
MEANWHILE, BACK IN COLLEGE ...
In higher education, efforts to measure online learning often treat learners as though they acquired the knowledge in a classroom, says Carliner.
The primary purpose of such measurements is to make a comparison to the classroom equivalent or for research purposes.
"I'm not sure that these are appropriate research opportunities to begin with," Carliner says. "Among other issues, it's difficult to make an assessment of learning value with a course that's only in its pilot phase, as many of these courses are.
"If the instructor hasn't validated the material, how can learning be assessed?"
OLL NEWS SAYS: Carliner's pre-conference session in Los Angeles Sunday, Oct. 17, is on how to evaluate online learning. Register at http://www.onlinelearning99.com.
AND OLL NEWS ASKS READERS:
Q What have you found to be an effective way to evaluate
online learning?
Q What DOESN'T work in evaluating online learning?
Q What's the most important thing for trainers to realize
early about evaluating online learning?
Please respond to [email protected]. Include your name, title, organization, what it does, where it is and a phone number at which we can reach you. Your subject line: Evaluating Online Learning.
_____________
PERFORMANCE SUPPORT WINNERS ANNOUNCED
Winners of the 1999 Electronic Performance Support Systems (EPSS) Design Contest are up in lights at http://www.epssinfosite.com/contest99/Contest_Results99.htm.
Honorees in the competition, sponsored by Bill Communications Inc.'s Performance Support '99 Conference and by the Performance Support Leadership Council, will receive awards at 5:30 p.m. Sunday, Oct. 17.
The kudos will come after Gloria Gery, conference chair, kicks off the co-sited Performance Support '99 and OnLine Learning '99 shows.
Register to attend at http://www.performancesupport.com.
_____________
WHO IS TRAINER 2000?
Here's an interesting exercise: What's the job description for the trainer of 2000?
Courtney Fontana ([email protected]), performance
consultant with Employee Performance Strategies Inc. (http://www.eps-i.com), a Chantilly, VA, training firm, says:
"Due to the onslaught of technology in training classrooms, you would think that training professionals would educate themselves on the basics needed to add value in the workplace. What skills does this new training practitioner need?"
Fontana's list:
o Basic facilitation and platform skills
o Communication skills
o Authoring, html, and Web-research skills.
ONLINE LEARNING NEWS ASKS: What would you add -- or subtract? And why? Respond to [email protected]. Please include your name, title, organization, where it is, what it does, and a phone number at which we can reach you. Your subject line: Trainer 2000.
_____________
MORE ABOUT RE-USING GAMES: HOW MUCH IS TOO MUCH?
Someone asked how much re-use of games in online instruction is too much.
Here's more of what you said:
"I can understand a vendor's desire to re-use an existing product, especially if it can reduce the cost to the client," says instructional designer Linda M. Watson ([email protected]), who lives and works in Rose Township near Flint, MI. "However, I have seen the 'one-size- fits-all' philosophy too often."
Ask these questions, Watson suggests:
o Will you get an actual discount if you re-use a game?
o Is the target audience the same or different? "If you are
continually using the same game with the same target
audience," warns Watson, "they are going to get bored, no
matter how good the game."
o Is there a better treatment for this subject?
One way to get a new game: "It may be time," Watson observes, "to look to a new vendor for a fresh approach."
Debby Kalk ([email protected]), a partner with Austin, TX, interactive-training developer Cortex Interactive Inc., defends re-use. "We encourage our clients to think of re-using elements creatively," says Kalk.
"It allows you to amortize your costs so that you can create more complex games," Kalk explains. "Or you can judiciously spend on the things people respond to most, such as graphics.
Learners may not even notice they're playing the same game as in the last learning module, Kalk claims. "Users actually don't notice structural similarities if the premise, graphics and media are different," she says.
"For a recent product, we created a single game format that required selecting the correct answer from a variety of possibilities. I think four or five of the 10 games used this same structure, but all had very different concepts and very different looks.
"Until we pointed out that the structures were identical, even our client didn't recognize this."
One note of caution: "Make sure that each use of the game is instructionally appropriate and effective," Kalk says, "and that its treatment is unique and compelling in each case."
Charles Miller ([email protected]), multimedia producer for Ernst & Young LLP in Cleveland, says vendors facing a new-or- used decision usually decide based by asking:
o What is the cost for development?
o What is the audience?
o What is the payback for a successful presentation?
"If the budget is tight, re-use is a good way to reduce the developmental costs," says Miller.
But if a training module has the potential for considerable return for the user organization, says Miller, "highly focused, original material is probably a good investment."
_____________
WHO'S GOING TO O.L.L. '99 AND P.S. '99?
Who's going to the shows? The full Show Guide database for OnLine Learning '99 and the co-sited Performance Support '99 show now is up at http://www.onlinelearning99.com/. Click Expo & Sponsors/Exhibitors.
The database includes contact information, exhibitor company descriptions, Web sites and more. "It gives a much more comprehensive picture of who's at the expo than the running list of company names that we've had thus far," says Steve Dahlberg, Bill Communications Inc. conference program developer.
_____________
WINDOW SHOPPING: FREE DEMOS, SALES HORROR STORIES
STREAMING COMPARISONS. Midi Inc., an interactive-multimedia design and development firm in Princeton, NJ, invites a look at interactive demos on the EyeOnIntegrity compliance and human-resources training Web site. The demos show streaming variations to accommodate bandwidth limitations.
DISTANCE-LEARNING DEMO. Rotor Communications Corp. of Los Angeles offers demos of its real-time distance-learning software at this site. The 15-minute interactive presentation shows how instructors can deliver content via network, intranet and Internet.
SALES THAT DIED. This site from a firm called Sales Autopsy Inc. of Streamwood, IL, lists "salespeople's worst nightmares."
OLL NEWS ASKS: Would any trainers volunteer to contribute cases to a site we could call "Web-Based Wipeouts?"
_____________
... AND FINALLY
*************************************************************
Hurry! Exhibitors, show your stuff at OnLine Learning '99
Oct. 17-20 in L.A. Check http://www.onlinelearning99.com.
*************************************************************
IT'S WEEKLY. IT'S FREE! To receive OnLine Learning News,
please go to http://www.emailpub.com/lakewoodand fill
out the form; or e-mail [email protected] with
"subscribe ONLINE-LEARNING-NEWS" in the subject line; or
go to http://www.lakewoodconferences.comand click: "Sign
up for two, free online e-newsletters!"
If you prefer NOT to receive OnLine Learning News,
please UNSUBSCRIBE at http://www.emailpub.com/lakewood.
Or e-mail [email protected] with "unsubscribe
ONLINE-LEARNING-NEWS" in the subject line.
____________
Copyright 1999
Bill Communications Inc.
