Y520 Home Indiana University

Link to
        Y520 Home Page

Link to
        Y520 Syllabus

Link
        to Y520 Class Schedule

Link to
        Course Units

Link to
        Y520 Online Conferences

Link to Web
        Resources

spacer

Unit 4: Descriptive Designs

Introduction
Lecture slides
Instructor notes
Readings
Learning activities
Web resources



Introduction

This unit briefly reviews the material we have covered this far in the course and then discusses descriptive designs.

So far our readings in the course have focused on the following questions. What is:

  1. inquiry?
  2. research?
  3. the quality of published research?
  4. theory?
  5. a model?
  6. the relationship of theory to research?
  7. a problem statement
  8. a hypothesis?
  9. a variable?
  10. a construct?
  11. an operational definition?
  12. a hypothesis test?
  13. a research design?

Inquiry & Research

Following the philosopher Elizabeth Steiner and conventional definitions, we stated in the first lecture notes that "inquiry" is "search for truth," and this goal is pursued by using a rational process for settling doubt. We stated that research, viewed as a process, is a thought activity; that is, it uses the rules of logic and publicly accessible methods that are applied to public knowledge.

The work of King, Keohane, and Verba (1994) was quoted extensively. To summarize, their view of research (the scientific method) includes:

  • The goal is inference,
  • The procedures are public,
  • The conclusions are uncertain, and
  • The content is the method.

We also noted briefly that this view of inquiry and research is not without criticism. Recall this comment reported by Schoenfeld (cf Lecture Notes 1):

By encouraging anything that passes for inquiry to be a valid way of discovering answers to complex questions, we support a culture of intuition and artistry rather than building reliable research bases and robust theories.

Contrary to many pronouncements that could be -- and will be -- cited later in the course, inquiry and research, as we have characterized them so far, follow specific procedures based on logic and rationality. One reason these procedures are important is, as Francis Bacon reminds us, "Man prefers to believe what he prefers to be true." It is so easy to see the "facts" that support our preferences and to overlook the counterindications. The scientific method is, to date, the best method we have, as we search for truth, to keep us honest in the sense that we do not just see the facts that we prefer to be true, but instead, that our observations and inferences correspond as closely as humanly possible to reality. The scientific method helps to protect us from falling into the trap that is the title of Thomas Gilovich's (1991) book, How We Know What Isn't So. Or, as stated by Pirsig in Zen and the Art of Motorcycle Maintenance, "The real purpose of [the] scientific method is to make sure Nature hasn't misled you into thinking you know something that you actually don't know."

Quality of Published Research

We were shocked and amazed -- well, at least some of us -- to discover that not all published research is of uniformly high quality. Wandt (1965, p. 6), to mention just one reading, reminds us "a large percentage of educational research articles published . . . contained serious flaws." In the following paragraph Wandt cautions: "The reader who turns to the research literature for assistance in solving a problem needs to know whether the conclusions reached in the articles he reads are probably correct. The only sound basis for determining the correctness of the conclusions lies in careful evaluation of the articles. Although there is a difference among journals in the quality of the articles they publish, the reputation of a journal cannot be used in lieu of the evaluation of a specific article. . . . There is no substitute for evaluation of published research."

In order to evaluate research, we must learn the conventions and standards for research.

Theory & Models

Kerlinger (1986) defines theory as "a set of interrelated constructs (concepts), definitions, and propositions that presents a systematic view of phenomena by specifying relationships among variables, with the purpose of explaining and predicting phenomena" (p. 9).

The purpose of scientific inquiry is first to understand a phenomenon, then to explain and to predict, and, if possible, to control the occurrence of a phenomenon. Ideally, theory integrates, organizes, and classifies the many facts documented in individual empirical studies. A theory explains by incorporating empirically established relationships among variables, and predicts by stating the theory's antecedent-consequent (cause-and-effect) relationships .

Theories are developed by combining logic with the results of observations. Theories may be expressed in such a way that constructs can be symbolized and the rules of logic applied to permit deductions. In other instances, a theory contains a model that depicts the relationships among constructs and permits conjecture.

Albert Einstein once said "It is the theory that decides what can be observed." Theory is what gives direction to an empirical investigation.

Relationship of theory and research

The process of theory development involves a continual refinement of the interaction between constructs and empirical observations in a manner that makes possible deduction leading to explanation and prediction. The input for theory development results from the research activities of observation, study, and experimentation.

Theories play several roles in the development of science. Theories:

  1. organize the existing knowledge in particular areas. A theory pulls together and enables "deeper meaning" by connecting the results of individual studies.
  2. explain the results of research studies. However, explanation in most areas is never complete and is subject to revision as more empirical evidence becomes available.
  3. predict future phenomena. Although theory is not an absolute truth, some theories are sufficiently developed to allow prediction with a high level of accuracy.
  4. enable control of occurrences of phenomena. Difficulty in manipulating variables limits inquirers; nevertheless, a goal of science is to enable control through theory.
  5. suggest new avenues for inquiry. Rather than being finished, theory generates ideas for further investigation.
The formulation and testing of a hypothesis related to the problem statement provides empirical observations that, combined with logic, are the basis for developing theories.

Problem Statement

Problem statement. Inquirers confront problems that puzzle and stimulate. These problems may be expressed as interrogatives, viz., Are fathers essential for positive child development? Do student-generated, higher-order questions facilitate comprehension? What is the effect of text summarization on classroom test performance? Does learning to reading faster improve the comprehension of poor readers? Are favorable course evaluations related to course work load? Do students feel they have learned more when the presentation is simple and easy to understand? Do they feel they have learned less when they must struggle with the material? Relevant theories, models, hypotheses, and empirical observations exist for many of these problems.

The empirical observations from a particular study are useful to the investigator who conducted the inquiry but may be of little use to other investigators unless these findings can be integrated into the larger body of knowledge. Hence, the empirical observations from individual studies are incorporated into theory.

Hypothesis

Popular opinion views theory as merely vague conjecture. This is partly correct: theory is conjecture, but theory must be explicit, clear, and unambiguous in order to generate specific, testable hypotheses. Operational definitions are the linkage between theory constructs (i.e., Shakespeare's "airy nothingness") and variables that can be measured. Hypotheses (or research questions) are developed to be tested; theories are developed to explain the empirical observations and to predict.

Variables

Examples of variables include anything that can be measured, such as school attendance, teacher expectancy, attitute of elementary education majors toward math, and reading proficiency. Variables are any attributes that we measure.

Constructs and Operational Definitions

The readings by Bolles (1967) and D'Amato (1970) both discuss the hypothetical nature of constructs and the necessity of specifiying the operations used to measure them. Recall the Bolles diagram that shows the linkages between the formal structure of theory and the empirical base.

Hypothesis Testing

Review the concepts of hypothesis testing that we have discussed thus far by visiting and reviewing the following two sites:

  Easton, Valerie J., & McColl, John H. (2000). Statistics Glossary.
Read the following entries:

  • hypothesis test
  • null hypothesis
  • alternative hypothesis
  • test statistic
  • significance level
  • p-value

  Chapter 9: Logic of Hypothesis Testing
Read the following entries:

  • Ruling out chance as an explanation
  • The null hypothesis
  • Steps in hypothesis testing
  • Why the null hypothesis is not accepted
  • The precise meaning of the p value
  • At what level is Ho really rejected?
  • Statistical and practical significance

References

Bolles, Robert C. (1967). Theory of Motivation. New York: Harper & Row.

D'Amato, M. R. (1970). Experimentation in psychological research. In Experimental psychology: Methodology, psychophysics, and learning. New York: McGraw-Hill.

Gilovich, Thomas. (1991). How we know what isn't so: The fallibility of human reason in everyday life. New York: Free Press.

Kerlinger, F. N. (1986). Foundations of behavioral research (3rd ed.). New York: Holt, Rinehart, & Winston.

King, Gary, Keohane, Robert O., & Verba, Sidney (1994). Designing Social Inquiry. Princeton, NJ: Princeton University Press.


Descriptive Designs

We recall that research designs can be classified as:

  • Description (includes all qualitative studies)
  • Correlation or relational
  • Experimental
  • Quasi-experimental (ex post facto and causal-comparative)
D'Amato discusses the first three designs in some detail. Why is it important to be able to identify the design used in a study? The type of design limits the strength of the claim that can be made concerning causation:
  • Description. Authors are justified in describing a phenomena. They may note certain variables that occur earlier in time appear to be (i.e., might be) related to variables that occur later, but this design does not justify causal statements.
  • Correlation. Authors are justified in stating that a relationship may exist between certain predictor and criterion variables, but the design and results from a single study do not justify stating that a causal relationship exists.
  • Experimental. If the study is conducted in such a way that threats to internal and external validity are minimized, then the experimenter is justified in stating that the independent variable is causally related to the dependent variable.
  • Quasi-experimental designs. This type of design is the most controversial. In some instances it is reasonable to infer causality. However, more often than not, investigators who use this design are not justified in making causal statements -- nonetheless, causal statements appear frequently.

Back to Top


When evaluating research articles, keep in mind Mueller's three questions:

  1. What do the authors want us to believe?
  2. What data address these (often causal) arguments/inferences/suggestions?
  3. Based on these data, how strong is the case for a causal argument?

Back to Top


Lecture slides


  Discusses the following desciptive designs: case study, focus group, meta-analysis, static group comparison, longitudinal (time series and panel).

  Contains suggestions for improving the weaknesses of selected designs for description

Back to Top


Instructor notes

    Press the button for notes. No instructor notes this time

Back to Top


Readings

Required:

Fraenkel & Wallen, Chapters 13, (15, 17)

Fraenkel & Wallen, Chapter 9

Back to Top


Learning Activities

None listed for this unit.

Back to Top



Web resources

None listed for this unit.

Back to Top


Last Updated: 01/02/24