What Clients Want

Karyn Romeis created an interesting post: Assessments in elearning responding to an earlier post of mine: State of Assessment by E-Learning Developers. This simple exchange has sparked quite a few different posts that I'll work on over the next few days.

Let me start with something I consider to be almost misleading in our industry. We hear all the time about how we SHOULD be doing Level 3 & 4. Will just said in his post Assessment Mistakes by E-Learning:

Stunning: Even after all the hot air expelled, ink spilled, and electrons excited in the last 10 years regarding how we ought to be measuring business results, nobody is doing it !!!!!!!!!

In Approaches to Evaluation of Training: Theory & Practice the authors state.

Evaluation becomes more important when one considers that while American industries, for example, annually spend up to $100 billion on training and development, not more than “10 per cent of these expenditures actually result in transfer to the job” (Baldwin & Ford, 1988, p.63). This can be explained by reports that indicate that not all training programs are consistently evaluated (Carnevale & Shulz, 1990). The American Society for Training and Development (ASTD) found that 45 percent of surveyed organizations only gauged trainees’ reactions to courses (Bassi & van Buren, 1999). Overall, 93% of training courses are evaluated at Level One, 52% of the courses are evaluated at Level Two, 31% of the courses are evaluated at Level Three and 28% of the courses are evaluated at Level Four. These data clearly represent a bias in the area of evaluation for simple and superficial analysis.

Maybe the authors didn't mean to say this, but they clearly say that 10% transfer is because of inconsistent evaluation. And clearly we are all biased towards the "simple and superficial." Yikes, I feel so dirty not doing Level 3 & 4.

But wait, we've all had the experience of when we try to do Level 3 & 4, we face considerable push back from internal clients and line managers. It's not that expensive to get the data nor really that much work, but it does take commitment. Obviously, the level of effort isn't worth it to the line managers. And this will be a topic for a future post, i.e., why don't clients care? But for know recognize that when you start with a client, you need to do a quick assessment of what they really wants:


Do they really care about effectiveness: changing behavior and driving business results? Many clients don't really care about this. They have a particular product/project in mind and they want you to get that done. As Karyn said in her post, the client will tell you, "Don't worry, your job is safe." They don't care about business outcomes. You'll often find this out pretty quickly when you start asking questions about "What do you expect people to do differently after this intervention?" or "What numbers are we trying to hit?" A blank stare often indicates that they don't really care. I actually think a surprising number of "training" projects involve clients who are on the don't care end of the spectrum.

Do they care about the looks of the product/project? Many clients come in with expectations about what will be produced and likely around what a "good looking" project will look like. They may be looking for highly interactive, or fun, or engaging, or games, or simulations, or ... If you suggest that a checklist might actually have better results than an hour-long, interactive training course, you may get lots of push-back. Some clients honestly don't care about looks, but most do. In fact, its much more common for a client to care about looks than to care about personality (I mean business results). Good looks are important in that it often implies user engagement, but this post is purely about assessing what the client really wants.

Remember that beauty is in the eye of the beholder. You definitely need to understand what they believe will be a good looking project. It may include what kind of format, graphic design, level of interaction, etc.

Also keep in mind that your assessment of what the client wants may also need to include thinking about other stakeholders (end-users, the client's boss, etc.) and what they want. Clark Aldrich recently suggested that helping your client get a promotion might be the ultimate goal. Thus, the "wants assessment" in that case needed to include assessing what your client's boss really wants. Do they care about business results? Do they want something good looking?

Finally, please, please, please do not show this graphic to your client (or ask the implied questions directly). Every client will tell you that personality is much more important than looks. Of course, they want it to be effective! Of course, it doesn't matter how good looking it is! They know that's the right answer! That's what they will tell you if you ask. Your job is to find out what they really think.

Stay tuned for more ...