State of Assessment by E-Learning Developers

Will Thalheimer posted a must-read Assessment Mistakes by E-Learning Developers which I think is mistitled. It should be the "State of Assessment." I'm not at all surprised by his findings in that they echo much of what I've heard over the years. Among them...

11% said they did NO evaluation
26% said they did Level 1 smile sheets
48% said they measured Level 2 learning
15% said they measured Level 3 on-the-job performance
0% said they measured Level 4 business results (or ROI)

His discussion further points out that doing Level 2 assessment immediately at the end of training (i.e., did a post course test ... likely multiple-choice) was not nearly as accurate as assessing later on.

The comments that follow are interesting, but I'm surprised that we aren't being more honest about this. I personally am a big proponent of using metrics throughout the solution (at analysis to define what it is we are really going after, as part of the intervention to provide direction, to assess the outcomes and make change), but often unless the metrics are already being collected by the organization (e.g., customer satisfaction surveys), then Level 4 will not happen. And, likely Level 3 - even though it can be approximated (cheaply) with a very simple survey won't happen unless those questions are already being asked other ways.

Why is it? Are we bad at our jobs? Do we not care? Hardly. But we must make choices about where we push. When you discuss a simple post performance intervention survey to assess how we did in changing performance with internal or external clients, the response you get is "they won't spend the time to do it" ... and they are generally right. Again, unless its part of the intervention itself (pre-post assessment) or part of the culture of the organization, this isn't a battle that's worth fighting for as compared to other battles.