The learn tech blogs to watch - all in one handy place!

Cross-posted to: GETIdeas.org

Whenever I think of assessment in the classroom, I am reminded of a rubric I created a long time ago to go along with a short writing assignment for my grade eight Language Arts class. You can see it below. Many would agree that rubrics are excellent tools — they show the students exactly what the expectations are and the scale we’ll use to assess their work. True. However, as a young teacher many years ago, I used this rubric alone and nothing else. I did not have an assessment and evaluation strategy to support my students and learn from my classroom practice. My students submitted work, and I evaluated it. That was it. What I’ve learned since is what every experienced teacher knows very well: that assessment for learning — an ongoing process that provides students with timely and meaningful feedback, informs us about how well the students are doing, and gauges the effectiveness of our classroom practice — must be an integral part of our role as classroom teachers.

Let’s look closely at my rubric. What exactly is it telling Terry about his work in my classroom and his grasp of the content we’re studying? Here’s one point of view: Terry received a rubric with detailed descriptors. If he takes the time to read them, he will see what he did well and what prevented him from getting a better grade. He can also see that he received a very good mark (80%) and a nice comment. Doesn’t this note contribute to Terry’s understanding of how well he’s doing in my class and how well he’s grasped the material? Perhaps. But here is another point of view: What does “Well Done!” really mean? What does it really teach Terry? How helpful is it in ensuring that he does well on the next assignment? Is it a scaffold that he can use to improve, to scale new and more challenging heights in Mr. Glogowski’s class? Is Terry (who is 13 years old) going to take the time to read the descriptors carefully? Does he truly understand how he achieved that A-, and how to repeat that success next time?

My point of view is that, based on this rubric, Terry now understands one thing very well: That assignment is now behind me, I’ve jumped through yet another hoop, and quite well. In other words, as a teacher, I missed an opportunity to engage Terry in a conversation about his work and his learning because the one thing that this rubric does very well — when used alone and not as part of a larger, more complex assessment strategy — is the following: It terminates opportunities for conversations with students about their work. The work is done, the grade assigned. It’s time to move on to another topic, another assignment, another hoop.

What if, instead of writing that short comment and assigning a grade, I took the time to write the following:

Terry, you took some risks with organization but it worked out beautifully. I know you know that topic sentences need to be at the very beginning, but by starting with an anecdote you totally pulled me in! Let’s chat about this next time we discuss your writing.

I also want to ask for your permission to share your work with the class tomorrow — there are some good examples here of how to be an effective storyteller. Is that OK? Talk to me if you think this would make you uncomfortable.

Finally, when I read this piece I see another important thing: Ever since you stopped giggling with Michael in class during writing time, your work has improved significantly. Did you notice that too?

Be ready to discuss your next draft with me tomorrow.

Is this assessment? I certainly think it is. It points out to Terry what makes his work good, provides loads of constructive encouragement, and shows that writing — and learning — is a process, one that involves conversations. Of course, in order to make this work, I would first have to ensure that the assignment requires several drafts, but that’s just basic common sense. How else is Terry going to become a good writer unless he understands that writing is a process? But what did my original rubric teach him? That school is about handing things in and getting a grade. My second attempt shows a very different conception of school and learning.

In too many classrooms, work is assigned, handed in, receives a grade … and any opportunity to engage students in thinking about and learning from their work is lost. In a classroom devoted to meaningful, timely, and effective feedback, and to assessment for learning, not mere assessment of learning, we engage students in conversations that provide them with the support and guidance they need to be successful. These conversations and the feedback we give also provide us — the teachers — with valuable information on how well we’re reaching and supporting the learners in our classrooms. And yet, in many classrooms around the world, assessment for learning is just not present, which begs an important question: what’s stopping us from providing this kind of ongoing and meaningful support to our students? Why is it so challenging to implement?

Derek Halpern, Social Triggers, February 7, 2012. What's worthwhile about this post is rthat it's at least honest about the technique, widely used (but seldom admitted) to drive readership. Called 'drafting', the idea is that you in some way associate your work with that of the famous and get drawn along in their wake. It's not just for SEO specialists like this one. It's for everyday bloggers - just list (or even better, link to) famous people (like from Noam Chomsky and Sir Ken Robinson, Mitch Resnick, Jaron Lanier, Conrad Wolfram, Ellen McArthur, Charles Leadbeater, Keri Facer, Stephen Heppell and Ray Kurzweil) and wait for the search hits to come in, or even better, for them to read your stuff and link to it. Or as Derek Halpern says, find your competitors, find the journalists who covered the story, and offer them a follow-up on the same story. Same concept. Or write things that praise the people who are already influential, and follow along in their wake. Same principle. [Link] [Comment]

Ross Dawson, Trends in the Living Networks, February 7, 2012. I should make a 'keynote speaker' video. I could record myself staring into space and mumbling incoherently. Every once in a while I'd shout "Yes! Fish!" [Link] [Comment]

Steve Wheeler, Learning With 'e's, February 7, 2012. Normally I wouldn't report on a conference set-up but these activity domes created by Graham Brown-Martin at Learning without Frontiers at Olympia in London caught my imagination. The presentations at the conference look decidedly ordinary, but as Steve Wheeler writes, "it was around and inside the domes that much of the conversations, connections and creativity took place. [Link] [Comment]

Diana G. Oblinger, EDUCAUSE Review, February 7, 2012. The current issue of EDUCAUSE Review focuses on leadership in the higher education information technology community. Diana G. Oblinger writes a brief overview of what constitutes leadership without missing a cliché:
- "Leaders catalyze change, not for the sake of change itself but for the sake of preserving fundamental values."
- "Values, experience, and analysis morph into leadership when combined through discipline"
Et cetera. My own view toward such 'leadership writing' is not kind. It seems to me that a lot of such writing is designed to play to the sympathies of people who are in leadership positions (much the way astrology columns stroke the egos of their readers) rather than raising more important questions, like (say) whether it's a good idea to have someone going around touting "fundamental values". [Link] [Comment]

David Wiley, Cable Green and Louis Soares, American Progress, February 7, 2012. David Wiley, Cable Green and Louis Soares seek to inject the concept of the open educational resource (OER) into popular perception in this article. The brief, intended for policymakers (and people who influence policymakers), urges "federal, state, and local governments and educational institutions to adopt a simple public policy: 'All publicly funded resources are openly licensed resources.'" Related: this backgrounder and coverage in the Chronicle (not that the Chronicle can really get behind any of this; alother article describes OERs as homemade digital alternatives. [Link] [Comment]

Audrey Watters, Hack Education, February 7, 2012. We may invent these things in Canada, but its people at places like Stanford who really know how to draw out that investment dollar. Hence we see another pair of Stanford professors, Daphne Koller and Andrew Ng, starting a massive open online course enterprise. Their startup, Coursera, looks a lot like Sebastian Thrun's Udacity. "We see a future where world-leading educators are at the center of the education conversation," says Coursera, "and their reach is limitless, bounded only by the curiosity of those who seek their knowledge; where universities such as Stanford, Harvard, and Yale serve millions instead of thousands." See also the Chronicle's Jeffrey R. Young's interview with MITx's L. Rafael Reif and Anant Agarwal. [Link] [Comment]

If you pit a good performer against a bad system, the system will win almost every time.

This quote from Rummler & Brache in Improving Performance, sums up many of the symptoms of hierarchical systems, whether they be schools, businesses or even prisons.

The great work to be done at the beginning of this century is the democratization of the workplace. Efficiency and effectiveness are not enough, and too often become mechanistic. It’s time to discard industrial management models that emphasize command and control and ensure that individuals at all levels have opportunities to engage in and question the system.

Without questioning, things can quickly go awry.

Gary Stager discussed the well-known Milgram Experiments, conducted in the 1960′s to see how far people would go in administering electric shocks to learners. These experiments were replicated by ABC News and Stager picked up the direct link to public education [please read the whole article]:

One of the subjects in the television program was a 7th grade teacher who explained that she didn’t stop shocking the learner because as a teacher she had learned when a student’s complaints were phoney. I thought to myself, “Has she electrocuted many students?”

The teacher asked the researcher, “There isn’t going to be any lawsuit from this medical facility, right?” When told that the teacher was not liable, she replied, “That’s what I needed to know.” It is however worth noting that this was after she induced the maximum shock and the learner demanded that the experiment be terminated.

This is why we need to change the entire education system – constraining curriculum; compulsory testing; useless homework; irrelevant subjects; classrooms cut off from the world; systemic bullying; etc. More or better teachers won’t help; we need to change the system.

In this interview, Dr. Philip Zimardo discussed the 1971 Stanford Prison Experiment, where students played their roles as guards or prisoners and abuses started within 24 hours:

But on the second morning, the prisoners rebelled; the guards crushed the rebellion and then instituted stern measures against these now “dangerous prisoners”. From then on, abuse, aggression, and eventually sadistic pleasure in degrading the prisoners became the daily norm. Within thirty-six hours the first prisoner had an emotional breakdown and had to be released, followed in kind by similar prisoner breakdowns on each of the next four days.

As Churchill said, “First we shape our structures, and then our structures shape us“. This reminds me of the question about who is the most important person on board a ship. Is it the Captain, the Navigator or the Engineer? Actually, it’s the Architect, because the initial design influences everything else.

Sometimes, no matter how hard you try, you cannot change the way things work in an organization. The problem may be the organizational model itself and it may be better to leave and create an alternative model than help keep a flawed one going.

Clay Burell had guest blogger Bill Farren discussing the hidden curriculum of school architectural design. He asked what hidden messages are our schools themselves asking by their inherent design:

• Did the building’s designers take into consideration its location?
• Who decided how (if) it should be built?
• Does the building make an attempt to connect students with their outside world?
• What does the formal, intentional curriculum teach?
• How is this formal, intentional curriculum taught?
• How is the school run?
• How is security portrayed?
• What is sold or advertised on campus?

There was an article I read many years ago, but never see cited, about designing learning environments. It’s Rodney Fulton’s SPATIAL model (1991) [my emphasis added]:

While a body of knowledge does exist that documents the relationships between learning and physical environment, there are problems that need to be resolved before the present level of understanding can be systematically advanced. One problem is that common vocabulary does not exist. Thus, in the literature, concepts are often described with similar but not identical terminology. Conversely, the same terms are used for similar but not exactly the same concepts. But this confusion in vocabulary is only a symptom of the fundamental problem: the lack of a conceptual model that explores relationships of physical environment to learning rather than to behavior in general. Architectural models address built environments, emphasizing both interior and exterior features of building design that allow, encourage, prohibit, or inhibit various behaviors. Psychological models discuss environmental attributes that set conditions for or even control human behavior. Sociological models emphasize the importance of environment in terms of how it facilitates human interactions. By emphasizing individual appreciation of the environment, aesthetic models address the relationship of values to human behavior. Workplace training models, including human factors engineering, emphasize the fit between environment and person and seek out optimal conditions for performance.

Each of these perspectives can add to a global understanding of the learning environment; however, a model that addresses learners in learning environments is a needed first step in refining educational research. The model described here – satisfaction-participation-achievement-transcendent/immanent attributes-authority-layout (SPATIAL) — can serve as a fundamental basis for organizing research designed to identify relationships between and among components of the learning environment and attributes of the learner. Further, this model has potential for weaving together findings from architectural, psychological, sociological, aesthetic, and human factors engineering studies.

Rodney Fulton responded, when I originally wrote this post in 2008:

I found it very interesting that some 17 years after I published the SPATIAL Model in a Jossey-Bass publication there was discussion that included the model. I am not aware of any significant use of the model or of any real impact on the field of Adult Education in the United States. I have longe since moved on from the field of Adult Education and am now very involved in Public Education at the Elementary level in the US. But again, it was gratifying to see my model referenced in 2008. If you know of any other people using or interested in the model, I’d be happy to hear from you. Thanks Rodney Fulton

There is still much structural work to be done.

Photo by Atelier Teee

Note: this post is an update of two previous posts from 2008

Mitch Smith, Inside Higher Ed, February 7, 2012. Long known for its Connexions service for building open educational reosurces (OERs), Rice University is now moving into the open online etxtbook business. "A free online physics book, peer-reviewed and designed to compete with major publishers’ offerings, will debut next month through the non-profit publisher OpenStax College." [Link] [Comment]

Many people are stuck in the world of linear, click-and-read content. Sometimes it’s because that’s all the organization wants. And sometimes it’s because we’re not quite sure how to make something more interactive. It’s probably the subject I’m asked about the most. One of the most frequent questions I get. The cool thing is that regardless of the tools you use, you can still build interactive content.

One of the best ways to learn to build better courses is to find some good examples, break them apart, and then try to build something similar. This way you get some hands-on practice, which is a lot more valuable than reading about interactive elearning.

So in today’s post, we’ll look at a popular example and talk through a few ideas on interactivity.

In large part interactivity requires a shift away from pushing the content out (which is common) and instead creating an environment where the learner pulls content in. A good way to get them to pull information is creating a situation where they need to make an informed decision.

I usually recommend throwing people in the pool. Instead of dumping them with a pile of information, dunk them in a pool of opportunity. Let them deal with real-life situations and learn through their decision-making.

Put them in the types of situations where they need to make the decisions you want to them to make. That creates the opportunity for them to demonstrate their understanding and if they’re deficient in understanding they’re motivated to pull the information they need to make an informed decision.

When I learned to create videos, I’d record TV commercials and then break them down scene by scene. Commercials are great because they’re very effective in communicating essential ideas and they’re short. It’s a lot easier dissecting a 30 second commercial than a 2 hour movie.

After viewing the commercial I’d create storyboards for each scene. Then I’d analyze the scenes, how they were edited, and consider the motivation from one shot to the next. This process of dissecting the commercial slowed it down for me and helped me see what was happening better. The same can be done with elearning courses.

Today we’ll look at a popular example that Cathy Moore shared a while back. We’ll do a simple break down of the module. This is a good example because it’s interactive, engaging, and it’s short enough to go through it a few times.

Click here to review the module first.

Basic breakdown of the course:

• Introduction. The course starts with an introduction that sets the stage and expectations. You can consider it the establishing shot. We want to establish what the learner’s seeing so that they know where they’re at and what’s expected.
• Context. After the introduction, they begin to build some context. They’re putting the learner in a real situation and then offering some guidance on getting through it. You’ll notice instead of a bunch of screens of information about the culture and country, they provide two characters who will provide differing perspectives. What I like about that approach is there’s a bit of ambiguity in the way they share their perspectives. This is much more like real life where things aren’t always tidy or obvious. It brings a healthy tension to the scenario.

• Challenge. Once the scenario’s set up the learner’s offered a challenge which is the course objective—make a good impression and build a relationship with the Haji. The scenario presents a series of mini challenges that build on each other. The squad leaders offer advice that can help with the decision-making.

• Choices. After collecting the advice a decision needs to be made. The course provides a few choices. One thing you’ll notice is that the choices are all plausible and viable. The problem with many interactive elearning scenarios is that they are very obvious and don’t deal with the nuances we face in real life. If all of the choices are viable it puts more pressure on us to pay attention and collect the right information. We want them to learn and not guess their way through the situation.

• Consequences. Each choice produces a consequence, with some more positive than others. Sometimes progress I made in the relationship and sometimes it’s a step backwards. I like the quick feedback because in lieu of a real conversation, you need to read the body language of the Haji. It’s enough of a tip without being too much feedback. Another thing I like about this particular scenario is that the choices aren’t all do or die. You can make a poor choice and still recover if you’re paying attention to the information you collect. Often we provide immediate feedback and that’s it. In this scenario you may find the right outcome but realize it wasn’t the best. It kind of motivates you to go back and find out what was a better outcome.

As you can see, breaking down the scenario is a great way to come up with a formula that could work for your own scenario.

• Introduction. Set expectations. Let them know what to expect and how long it should take. It also helps if the course is visually engaging.
• Context. Put the learner in a real world environment where they make the types of decisions that impact their performance. Not sure what that is? What’s the goal and expected outcome after the course? How can they demonstrate that they’re able to meet the objectives?
• Challenge. Give them some good challenges that get them to think. You can even add a few distractors. Some people like to jump ahead and answer questions first and wait for feedback. But others will want a bit more information to make an informed decision. Give them the freedom to do so.
• Choices. What choices do they have to work through the scenario? Make them viable and real. If they’re obvious choices, then the interactivity is wasted. Sometimes I throw them for a loop by making all choices wrong or all of them correct. Not having an “all of the above” or “none of the above” option adds some healthy tension.
• Consequences. Each decision produces consequences. Sometimes the consequence is simple feedback and sometimes it can become another decision-making challenge. Do this to vary the pacing. You don’t always need to provide immediate feedback. Delay it. You’ll notice that in the scenario there’s some feedback from the Haji to give you a sense of what direction you’re moving, but it’s not “right” or “wrong” feedback. You just have to keep plodding forward and you’ll find out if you succeed at the end.

Without too much focus on the great visual design of the course, take note of how you can navigate through the module. There’s always a place to restart and also quickly advance through the modules. This gives the learner freedom and control. It also encourages going through the course a few times. My guess is that most people go through the module at least twice if not more. Would that be the case if every screen was locked like they are in many courses?

Cathy discusses the scenario in more detail on her blog. But before you look at it, do this. As you go through the course try to map out the flow. How many individual screens are there? Go through it a few times. You’ll start to see a pattern and realize that the structure of the program is not that complicated and easy enough for you to do, regardless of your authoring tool.

The secret is taking the time to analyze the course and then creating a model that you can repeat with your own content.

Here’s another cool scenario-based module that’s been making the rounds. It’s a good one to dissect. Do the same thing; make some notes on what you like and how it flows. Try to create your own outline of the course. Then use that structure as a guideline for your own scenario.

Click here to view the interactive scenario.

If you want to look at other multimedia examples for inspiration, David does a great job collecting them at his elearning examples site. Most of them are smaller interactions so they’re perfect to break apart.

Perhaps you can take one of your linear courses and convert it to an interactive scenario using the same structure and outline as the soldier scenario. It’ll be good practice and I’m sure you’ll wow them at work.

If you do rework a course and make it more interactive, feel free to share the link. We’d love to see what you do.

Tidbits

I just added the information for the elearning workshop I’ll be doing in London on May 22. It’s going to fill up quickly, so make sure to sign up before it’s too late.

Also added details for the elearning workshop in Hampton, VA on April 17.

Upcoming Events

• February 13-14: Atlanta (Training Magazine). Swing by the Articulate booth to check out the soon-to-be released Storyline. Here’s how to get a free expo pass.

• February 22-23: St. Louis (ASTD). February 22: Rapid E-Learning Workshop and February 23: PowerPoint Doesn’t Have to Be Boring. We’ll follow the presentation with a free Articulate jam session.

• March 21-23: Orlando, FL (Learning Solutions Conference)

• April 10: Jacksonville, FL (NEFL ASTD). Details coming.

• April 17: Virginia (SEVA ASTD). Rapid elearning workshop. The first half we’ll look at some basic course design and in the second half we’ll learn to build interactive content.

• April 5: Portland, OR (OpenSesame). An E-Learning Heroes Roadshow workshop where we’ll look at ways to build elearning courses and learn some great PowerPoint tips. Excellent price and great coffee!

• May 17: Orange County California. Details coming.

• May 22: London, UK. I’m doing a full-day elearning workshop. I’m looking forward to meeting the blog readers in the UK.

• May 24: Leeds UK. This is the annual Articulate users conference hosted by Leeds. It’s a great way to connect with other Articulate users.

• October 4: Seattle, WA (ASTDPS). Details coming.

• October 17: Bloomington, IL (CIC-ASTD). Details coming.

Michael Trucano , EduTech, February 6, 2012. Interesting first-person reflection from a staffer at the World Bank on mobile learning in developing nations. "A long-anticipated new era of hype is now upon us, taking firm root in the place where the educational technology and international donor communities meet, with 'm-' replacing 'e-' at the start of discussions of the use of educational technologies." And here I thought the next big letter would be r-. Oh well. I now predict that the next big letter (after e-, i-, and m-) will be p-. He continues, "I do often feel that many of the discussions around 'm-learning' end up sounding a lot like general discussions of ICT use in education... (but) I do think there is something fundamentally different about the potential for mobile devices. My hope is that, given all of the groups now considering this an increasingly important priority area for action of some sort, in 2012 practical insights into what this mobility might mean for both educators and learners based on real life experiences."
[Link] [Comment]

Judy O'Connell, HeyJude, February 6, 2012. It's easy to describe ten 'megatrends' in such a way that most people would nod in agreement, I think, but it's hard to get them precisely right. To take example that is a bugbear of mine, consider this one: "The world of work is increasingly global and increasingly collaborative." This isn't quite right. The world isn't "increasingly collaborative" - if anything, it's less so. But what collaboration there is has gone global. But that means that in your day-to-day world you will experience less collaboration with those around you - how do you get by, then? Perhaps by dog-eat-dog competition for local resources, but more likely by cooperation - pooling (for example) purchasing or production power, but not for the same ends, but for distinct ends. And indeed, if we look at it that way, and recognize that when "teams (are) geographically diverse (and) are also culturally diverse" what we understand by collaboration changes. If you think your work group has simply gone global, as this 'megatrend' suggests, you've misinterpreted this trend in a major way. [Link] [Comment]

Hans de Zwart, Technology as a Solution…, February 6, 2012. When I was in Australia in 2001 I would from time to time (ie., frequently) find myself in the local public house. At one of these there was this novel phenomenon called "speed-dating" taking place upstairs. The idea is each prospective partner would interview others for short three-minute periods, and then at the bell move on to the next, taking note for later those who seemed the most interesting. It seemed so intriguing and I was eager to try it, but having been recently married decided it would be inappropriate. So I've never had the experience. But the technique applied to a conference event sounds more than fascinating, and Hans de Zwart has done the concept justice with this wrap-up of his speed-conference experience. It makes me wonder what an online version would look like - it would have to be better than match.com, right? [Link] [Comment]

Tony Hirst, OUseful Info, February 6, 2012. While I believe that evidence is crucial to decision-making, I am sceptical about "Evidence Based Policy Making" (or "evidence-based government" or "evidence-based education", etc.). Why, despite the apparent contradiction? Because the one is not the same as the other. In the former, you look at various claims from all sides, weigh the alternatives, take into account values and circumstances, and act on the basis of a reasoned decision. In the latter, you are led blindly by "the evidence" as presented, where (as Tony Hirst suggests) "'evidence' inherits the authority associated with the most reputable source associated with it when we wish to call on it to justify it." "Evidence-based..." is often, in other words, a mechanism used to disassociate decision-making with evidence and reason, and to instead stamp authority with the imprimatur of 'evidence'. Hirst offers a good examination here, and an equally good follow-up. [Link] [Comment]

Helge Scherlund, Elearning News, February 6, 2012. I've been think of doing something like this for MOOCs (I may well still do it, as I have a presentation scheduled in that direction in a few weeks). I think I'd want it to be a bit more practical and process-oriented that this item (the actual checklist is on page 11). [Link] [Comment]

Mark Wagner, Educational Technology and Life, February 6, 2012. This article contains seven good tips and three plugs for commercial products. If you don't mind that, then it's a good primer for getting started in social networking. [Link] [Comment]

Giulia Forsythe, gforsythe.ca, February 6, 2012. Not that it's necessary, I think, but I guess people should know I'm continuing my years-long boycott :) of services like Elsevier. The boycott, as the diagram shows, covers publishing, refereeing, and editorial work. But here's the challenge I have for academics: will you also refrain from reading and citing Elsevier journals? Ah, too hard!? [Link] [Comment]

James Meikle, The Guardian, February 6, 2012. If Twitter is harder to resist than alcohol and tobacco, why do I have to force myself to remember to check to see whether anyone has written a message to @downes? (If I miss your comment, I'm sorry, I hardly use Twitter, and use it less and less as time goes by.) Related: Dave Winer on country-specific Twitter filters: "We should have tutorial sessions at every Internet policy conference that show people how easy it is to operate your own infrastructure. It's really there now, ready to teach users how to do it. But you have to make a commitment to standing up for the Internet. It will never be as easy as Twitter. However, if Twitter shuts you off, it won't effect your presence. That's worth a little more complexity." Amen. [Link] [Comment]

Jenny Mackness, Jenny Connected, February 6, 2012. Interesting article with the backstory to an IRRODL article on MOOCs referenced here last week. I love the bit about the reviewers ("We didn’t receive any guidance from the Editor as to which Reviewer to believe. So we didn’t do a major rewrite :)"). Also worth noting: "Reviewer B strongly objected to our use of blog posts as sources of information, and I have to say that we rather strongly objected to his/her objection." Mackness gives three very good reasons for her position:
- most of the conversations about connectivism and MOOCs happen in blogs
- we were worried that our paper was going to be out of date before it was even published
- neither of us works for an academic institution, nor do we live within easy access of a university library
I'm sympathetic. Most of my work has been published in blog form; from my perspective life is too short to have to deal with arbitrary reviewers and edits well past the point of diminishing returns. The result has been that the citations have frequently gone elsewhere. I understand the need for peer review - but we need a better system. Realted: MOOCs are here to stay, by Graham Attwell. [Link] [Comment]