School of Education Building          Fall 2007

EDUC-R690: Application of Research Methods to IST Issues

Credits:  3
Class hours: Wednesday, 1:00 - 3:45 p.m. in ED 2275
Instructor:  Ted Frick
Office and phone number:  ED 2218, (812) 856-8460

Table of Contents

Goal and Objectives

The goal of R690 is:

To help you learn disciplined inquiry in Instructional Systems Technology through first-hand experience -- i.e., by by doing representative research tasks and critquing research done by others. 

Objectives
  1. Conduct interviews for needs assessment (qualitative method).
  2. Do content analyses (qualitative method).
  3. Conduct usability evaluations (qualitative and quantitative methods, problem diagnoses).
  4. Analyze existing survey data with SPSS (quantitative methods, tool skills).
  5. Make a one-page Web survey, collect and store data via Transform, import data into SPSS, label variables and values (tool skills).
  6. Critique research reports done by others.

We will only have time to do an introduction or overview of most of these tasks in this class. But you will get your hands on data, do data analysis by hand and with computers, and even collect some data yourself. Similarly, you will critique research articles in the IST field, but this will be at an introductory level.

You will take inquiry courses where you can study and apply these techniques in greater depth; you will likely have opportunities in working with faculty mentors in R695 seminars to use some of these techniques and tools skills and to refine them; and of course you will do research yourself culminating in a dissertation study at the end of your Ph.D. program in which you will very likely use one or more of the above techniques.

Overview of the Course

1.  Do a needs assessment of stakeholders for the School of Education Website. This will involve collection and analysis of mostly qualitative data from several sources.

1.1.  Interviews of School of Education students, faculty and staff.  Once we have a set of interview questions (which we will develop together in class), each R690 student will conduct two interviews and write up results (this a deliverable for R690).  R690 students will work in pairs; each one does each interview with his or her partner as an observer.

1.2.  Collect FAQs from information gatekeepers. What are the 10 most frequently asked questions:  how often; who asks, when asked, how asked; and what is the answer to each FAQ.

1.3.  Do content analysis of results from 1.1-1.2.  This will be a class activity led by the instructor. Results of this content analysis will be recorded and shared with the clients.

2.  Do usability tests of the current School of Education Website. 

2.1. Use content analysis results from 1.3. to identify tasks for the usability tests. R690 students are paired.  Each pair conducts two usability tests and writes them up (this a deliverable). 

2.2.  Do a content analysis to identify usability problems with the current SoE Website.  This will be a class activity, resulting in a table of tasks, observations, and problems.  This table will be shared with the clients.

None of the inquiry tasks above will require human subjects approval by the IRB, since these are class activities for learning purposes, and the results will not be published.  This is not research resulting in generalizable knowledge or results to be disseminated, according to the definition of research in the IRB requirements. 

The basic goals here are for students to practice interviewing skills, practice doing content analysis for synthesis, practice doing a usability test, and doing yet another content analysis in the problem identification. These are largely qualitative methods, but applied to an IST kind of issue -- i.e., how to improve some kind of technology product.

3. Analyze existing Web survey data.

The 3rd project for R690 will involve analysis of Web survey data (nothing to do with SoE Web site).  Students will be analyzing some existing survey data (already collected in a previous study -- with data on 190+ subjects, which is sanitized), in order for students to get practice in doing basic descriptive statistics and their interpretation. 

Each student will be given a set of analysis tasks as a deliverable. Each student will work with a unique subset of about 15-20 cases to be done via SPSS and is expected to interpret the results in an individual written report (with tables, figures, and interpretations of results). For specified variables, the analysis tasks will include use of the appropriate descriptive statistic and interpretation (frequency distribution, central tendency, range, dispersion). Several relationships between variables will be described using the appropriate measure of association (e.g., Pearson Product Moment correlation, chi square, Spearman's rho). In addition, you will do a pattern analysis for combinations of 3 or more classifications.

4. Build a one-page Web survey with HTML, use Transform to store the results, administer it to at least 5 classmates or friends, import data into SPSS, and label variables and provide appropriate variable types and value labels as needed for variables.

Each student will create his or her own little survey on the Web (one Web page with several different question types, using HTML and Transform), collect some pseudo data with it via R690 classmates or friends, and import the collected data into SPSS for analysis. The deliverables are:

4.1. The SPSS .sav file with the sample data in it with variables named and coded as needed, and

4.2. The HTML Web page with the survey questions (no analysis of pseudo data is required, however).

These are largely quantitative methods in projects 3 and 4.

5. Choose an existing research study in an area of interest in IST, present a summary of the research to the class, and lead a discussion that critiques the study.

The presentation and discussion during an R690 class meeting will last approximately 45 minutes. The presentation should be done in Powerpoint, and include:

5.1. A summary of the purpose of the study, how or why it is relevant to IST, research questions addressed, methods used, and results or conclusions of the study by its author(s).

5.2. Identify major strengths and weaknesses of the research study according to criteria discussed in class, as well as in Fraenkel and Wallen (2006) for the type of research that is attempted.

6. Write a formal critique of that research study, using APA style.

The critique should include a:

6.1. Description of the purpose of the study, how or why it is relevant to IST, research questions addressed, methods used, and results or conclusions of the study by its author(s).

6.2. Description of each criterion used and justification of it according to philosophic principles and/or accepted values held by research methodologists regarding the respective research method(s) used in the study being evaluated.

6.3. Application of each criterion to the study being evaluated and description of how the criterion was or was not met.

About Plagiarism: Do not do it!

Originality of Work: all work you submit for R690 must be largely your own. Plagiarism will not be tolerated. If you plagiarize the work of others, you will receive a failing grade for R690 and be reported to the IST chairperson for disciplinary review. If you do use the work of others, it must be a minor portion of what you submit and the original creator must be fully credited and such credits must be clearly visible to the user.

Evaluation and Grading

Grading Scale

 

Readings

Fraenkel, J., & Wallen, N. (2006). How to Design and Evaluate Research in Education with PowerWeb (6th ed.) New York: McGraw-Hill. (required -- just order it online for yourself, e.g., at Amazon.com)

Steiner, E. (1988). Methodology of Theory Building. Sydney: Educology Research Associates. (available online for R690 students).

Additional readings are listed in the R690 schedule and resources (IU network ID and password required for the latter).

Department of Instructional Systems Technology
School of Education
Indiana University Bloomington

Last updated:  October 22, 2007 by T. Frick
Copyright 2007, The Trustees of Indiana University