Introduction:
Hints for successful learning:
- Follow unit two's objectives
- Read through the reading resources, which are extensive in this Unit
- Understand each key concept
- Do the practice questions and assignments
- Participate in online activities, including chat and practice question
postings
Learning Goals - At the end of this unit's materials, you should be able to:
- Define the term "reactionnaire"
- Describe the different purposes and uses of reactionnaires
- Describe the strengths and weaknesses of reactionnaires
- Describe categories (dimensions) of questions
- Construct reactionnaire items (questions) and response categories
- Apply general guidelines for good reactionnaire layout
- Analyze and present Level 1 evaluation data
Readings
Required:
Electronic reserve documents found at http://ereserves.indiana.edu/eres/coursepage.aspx?cid=3582
- Phillips, J. (1997). Handbook of Training Evaluation and Measurement Methods (Improving Human Performance Series) 3rd Edition. Butterworth-Heinemann. Chapters 7 - 9.
- Lee, S. H., & Pershing, J. A. (1999). Effective reaction evaluation
in evaluating training programs: Purposes and dimension classification.
Performance Improvement, 38, (8). 32-39.
Explains the basic uses and types of information gathered
using reactionnaires. Explains the importance of carefully selecting
evaluation dimensions and rigorously designing and developing reactionnaires.
Located in the R561 electronic reserve.
- Pershing, J. A., & Pershing, J. L. (2001). Ineffective reaction
evaluation. Human Resource Development Quarterly, 12 (1). 73-90.
Summarizes a research project that evaluated fifty
reactionnaire forms that were regularly administered by a prestigious
medical school. Four areas of survey design and development were examined.
Results indicated that a thorough assessment of currently used reactionnaire
forms is likely to locate weaknesses and that appropriate revisions
would benefit evaluators, instructors, and learners. Located in the
R561 electronic reserves.
- Pershing, J. A., & Lee,
S. H. (1999, Draft). Analyzing needs for performance improvement:
Processes and core competencies. Section 14.
A detailed explanation of the types, elements, construction,
and layout and formatting of questionnaires. Explains the advantages
and limitations of questionnaires and provides information about selecting
respondents, administering a questionnaire, and analyzing data as well
as reporting findings. See Issues on
PDF Files for assistance with this file.
- Pershing, J. A., & Lee, S. H. (1999,
Draft). Analyzing needs for performance improvement: Processes and
core competencies. Section 18: Graphing Data.
A detailed explanation of the types, elements, construction,
and layout of graphs and tables. Explains the advantages and limitations
of using several types of charts and graphs for displaying data. See
Issues on PDF Files for assistance with this file.
- Pershing, J. A., & Lee, S. H.
(1999, Draft). Analyzing needs for performance improvement: Processes
and core competencies. Section 10: Document Analysis.
Presents an in-depth explanation of the data collection
technique of document analysis. Explains the purposes and uses of document
analysis and the various methods used. Provides information on reporting
document analysis findings. See Issues
on PDF Files for assistance with this file.
- Pershing, J. L. (2002). Using document analysis in analyzing and evaluating
performance. Performance Improvement, 41(1). 36-42.
Provides a rationale and overview for using document
analysis in analyzing and evaluating performance improvement initiatives,
including training. Provides specific examples of the document analysis
techniques of tracing, content analysis, and case study aggregation.
Provides a graphical and step-by-step explanation of the document analysis
process. Located in the R561 electronic reserves.
Optional:
- Birnbrauer, H. (1996). Improving evaluation forms to produce better
course design. Performance & Instruction, 35(1), 14-17.
An extension of the rationale for using reaction evaluations
within training: provides some specific ideas about form design as well
as effective and ineffective question and response formats. Located
in the R561 electronic reserves.
- Kirkpatrick, D. L. (1998). Evaluating reaction. Chapter 4 in Evaluating
training programs: The four levels, 2nd edition. San Francisco: Berrett-Koehler.
Donald L. Kirkpatrick is the inventor of the Four Levels
of Evaluation. This chapter explains his perspectives on reaction evaluation,
which he labels level 1 evaluation. He provides guidelines, forms, procedures,
and techniques for conducting a reaction evaluation effectively. Located
in the R561 electronic reserves.
- Pershing, J. A., & Lee, S. H. (1999, Draft). Analyzing needs for
performance improvement: Processes and core competencies.
- Pyrczak Ch. 4-6, 8-11, 15, 23 -or- Other basic statistics textbook
Advance Organizers:
- Imagine you are the training manager for a large corporate university. Your university has an extensive course catalog, but enrollment has been declining steadily during the past few months. Your superiors are beginning to question the low enrollment and want you to determine why it is happening. You begin to wonder whether employees are not pleased with the courses or if there could be some other organizational factor at work. You decide that to help answer that question, gathering data on trainees' reactions to the courses is necessary. This spurs a variety of questions: Are there advantages and disadvantages to incorporating employee reactions to training as a part of an overall evaluation scheme? What kinds of data must be gathered from training participants, and how can the data be summarized for others (i.e. superiors) to view?
- You have just attended a seminar. A sloppy, confusing questionnaire (in your humble opinion) was handed out at the end. You begin thinking to yourself: What makes a question good or bad? What impact do poorly worded questions and answers have on the usefulness of the information gathered? In other words, if I didn't know how to answer, would anyone else? How should questions be related to the original objectives of the training? And, what must be done to insure a clean crisp questionnaire layout? Maybe there are some good books on the subject…you make a note to check out possible books in the library.
Next Section
