Objectives and Evaluation, Part 1
Levels of HRD Evaluation

Popular wisdom states that we need to define outcomes before we begin.  One version states “If you don’t know where you’re going, you won’t know when you got there;” another states “If you don’t know where you’re going, you’ll end up somewhere else.”   Whichever version you prefer, the point is the same: we need goals to provide direction—and to provide a basis for measuring our progress. 

Most organizations have two sets of goals.  Official goals define the mission of the organization, what it values.  Operating goals define the specific, measurable objectives that the organization is seeking to achieve. Thus, a university committed to “career preparation” would set goals ensuring the employability of its graduates and would evaluate its programs in terms of job placements and salaries of recent graduates. A university committed to “intellectual development” might set very different operating goals, and might measuring the percentage of students accepted into graduate school, their success in graduate school, and the quality of the programs in which they enrolled.

And you must be able to prove it.   The HRD unit which cannot prove its unique value to the overall organization will not flourish.  The HRD consulting firm which cannot prove its value to its clients—potential and recent—will likewise suffer.  A leadership development program is not only evaluated by the participants who attend, it is also evaluated by the organizations who pay for their members to attend. 

Donald Kirkpatrick has described four levels of evaluation: Reaction, Learning, Behavior, and Results.  Jack Phillips has recommended a fifth level, Return-on-Investment.

“Reaction” is the easiest level to assess, whether we ask whether participants “liked” the seminar or if they felt that the facilitators were knowledgeable and caring.*  It is very easy for participants to fill out a form asking them whether they liked the program or not; it does not require thought and participants clearly see that it is the facilitators who are being evaluated, not the participants.  When we measure reaction, we are measuring opinion, not learning.  

“Learning” is more difficult to achieve and to measure than “Reaction.”  Although you have had the good fortune to attend a university where faculty know precisely how to measure what you have learned in a course, this is not typical.  Successful measurement of “Learning” begins with clear objectives, statements of what the student is to learn.  Those objectives should determine program design and assessment. 

Changes in “Behavior” are even more difficult to achieve and measure.  How many times have you learned something new and said “that’s good information, I’m going to have to act on that” but then failed to act?  Sometimes we forget the information; sometimes we cannot quite figure out how to apply the information; sometimes we try it once, get less-than-perfect results, and forget it.  Behavior requires practice, which can be time-consuming.  And how do we measure the behavior of people after they have left our field of observation?

Think carefully about this one. Suppose that Acme Industries sends you three people who have great task skills but whose careers are stalled because they lack people skills.  Acme pays for them to attend your program, the attendees enjoy your program and learn your theories, but they remain as insensitive as they were before the training.  Will Acme pay for additional people to attend your seminar?  Whether you are an outside vendor or working within Acme, your credibility is shot.

Changes in “Results” may or may not be easy to measure, but they are difficult to prove.  Perhaps you promise participants that they will raise their 360^o  feedback ratings as a result of your workshop. You are not working in a controlled laboratory, so other factors may affect the results you are measuring.  And those evaluating your program will immediately think of some of those other factors.

“Return-on-Investment” is somewhat more difficult.  You must not only prove that your workshop has produced results but that the results contribute toward organizational goals and were worth the investment.

There are several sound reasons why your participants should learn to regularly assess their own abilities and compare those to the ideal state.  The most obvious are those we described when discussing organizational goals: they set direction and they can be used to measure progress.

But Assessment is also important as a motivational tool.  Many of you are somewhat familiar with David C. McClelland’s Learned Needs Theory and his studies of the Achievement-Motivated Personality. Achievement motivated people strive to meet goals which are specific, observable, and measurable; and which are challenging but achievable.   And most of us increase in our Need for Achievement if we can track our progress in achieving important, challenging , achievable goals. 

What about Objectives? (Click Here)