Evaluative Summary of Articles on Nested Design
Submitted by Allan Wei
I.D.

1. Background Information
Author(s): Fox. Roy F
Title: Treatment of Writing Apprehension and Its Effects on Composition
Source: Research in the Teaching of English
Date 1980. February. 39-49
 

2. Abstract

This study deals with the effects of student-centered methods of teaching composition (workshop) on writing performances, including reducing writing apprehension, quantity of written products (number of words), and writing quality. It was hypothesized: (1) that student-centered methods of teaching composition would help students reduce writing apprehension more than traditional methods (teacher-centered methods), (2) that student-centered methods would help students produce more words per essay than traditional methods, and (3) that student-centered methods would help student create better quality of writings than traditional methods. A partially nested design was used for this investigation. A total of 106 freshman students of six intact classes were used as subjects. Three graduate instructors taught one experimental and one control section each throughout the 16-week period.

The results suggest three main conclusions. First, the experimental groups were significantly lower in writing apprehension than the control groups at the end of the 16 weeks. Second, the experimental group writers produced approximately 49 more words per essay than control group writers. Third, the experimental treatment produced writing at least as proficient in overall quality as the writing produced by conventional composition instruction.

3. Null hypothesis, level and sample size per group.

Eight detailed hypotheses were stated in the report, but they can be restated as the following three major hypotheses:

Hol: There is no significant difference in student writing apprehension between student-centered methods of teaching composition (workshop) and teacher-centered methods of instruction (tradition).

Ho2: There is no significant difference in student written essay length between student-centered methods of teaching composition (workshop) and teacher-centered methods of instruction (tradition).

Ho3: There is no significant difference in student writing quality between student-centered methods of teaching composition (workshop) and teacher-centered methods of instruction (tradition).

A .05 alpha level of significance was selected for all analyses of data. A total of 106 freshman students of six intact classes were used as subjects. No further information shows the sample size of each class. Given the ratio of students to classes (106/6), we can surmise that the sample size was not equal in each class.

4. Independent and dependent variables

Independent variables:

(1) Methods of teaching (conventional and workshop)

(2) Instructors (three different instructors)

(3) Classes (six classes nested within treatments)

Dependent variable: Writing performances with three levels: (1) writing apprehension

(2) average number of words per essay

(3) writing quality

5. Instrument, briefly comment on its reliability and validity.
  1. In the traditional instruction classes, all response to student writing came from instructors. Control group instruction in essay organization focused on traditional rhetorical modes. Other conventional methods included lecture, instructor-led discussion, question and answer sessions, and writing exercises.
  2. The student-centered methods of instruction (workshop) involved large-group interaction exercises, paired-student and small-group language problem solving activities, free writing, practice responses to writing, structured peer response to writing, specific objectives for each essay, and instructor-student conferences.
  3. The Writing Apprehension Test, administered at the beginning and the end of the experiment, is a 26-item scale used in numerous research studies since its initial testing. No information about the reliability and validity of the test was given.
  4. The post-test writing sample administered at the end of the study was controlled across conditions for time, topic, type of writing elicited, time allowed for completion, and procedure of administration. Essays were holistically scored by two trained and experienced raters. Raters discussed scores that differed on more than two points. The Pearson Product Moment Correlation Coefficient on the raters' adjusted score evaluations was .92. The criteria for evaluating essays were not presented.
6. Experimental Procedure (brief)

One hundred and six freshman students of 6 intact classes enrolled in English Composition at the University of Missouri-Columbia during the spring semester of 1978 were used as the subjects of the study. Three classes were assigned to the experimental group (student-centered methods), and 3 to the control group (teacher-centered methods). All classes met for fifty minutes, three times per week. Three graduate instructors taught one experimental and one control section each throughout the 16-week period. Prior to this study, all three instructors had been exposed to the theory of workshop instruction, but had no direct experience using it in their classes. They had experience with the traditional teaching methods. The 3 control groups each completed five outside-of-class essays, along with numerous in-class paragraphs and exercises. The 3 experimental classes each completed seven out-of-class essays along with one free writing per week and occasional in-class writing exercises. For assigned essay topics, both groups selected from the same pool of topics. Two instruments were used to gather data: the Writing Apprehension Test and a two-hour post test writing sample.

7. Statistical analysis and conclusion.

This study should be a partially nested design because it used intact classes. Classes are nested within teaching methods and instructors, and teaching methods and instructors are crossed. However, the researcher treated the study as a 2*2 factorial design and neglected the nested factor. Therefore, the ANOVA tables in the report were not correct, thus generating some incorrect conclusions. Nevertheless, if based on the report, the following conclusions were obtained:

First, students' fear and avoidance of writing and of having their writing evaluated can be significantly reduced using either method investigated

Second, the sequential and largely student-centered experimental treatment significantly reduced writing apprehension at a faster rate than conventional instruction.

Third, composition instruction that is similarly structured can produce writing at least as good as that produced in conventional composition classes.

Fourth, there were no interaction between instructors and treatment. Thus, provided with only two-hour weekly training sessions, instructors became sufficiently skilled to apply the experimental treatment with consistent results.

Finally, the conclusion about the length of essay is erroneous. Since the significant alpha level was selected at .05 beforehand, the writer distorted the results to favor his belief (The difference was significant only at the .056 level of confidence [see Table 4]).

8. If you were the researcher, how would you improve the study?

If I were the researcher, I would do the following to improve the study and the report:

a. To correctly define the dependent variable for the study. The dependent variable should be writing performances with three different levels -- writing apprehension, length of essays, and writing quality, not three different dependent variables.

b. To take the nested treatment into consideration. The nested treatment is the 6 classes. The nested factor effects should be included in the ANOVA tables.

c. To specify the sample size of each class and try to use equal sample size for the intact classes.

d. To attach sample rating criterion for evaluating the writings at the end of the report.

e. To clearly describe the validity and reliability of the Writing Apprehension Test.

f. To clearly describe the detailed process and administration of the Writing Apprehension Test.

g. To draw conclusion for the study on the basis of the alpha level specified beforehand (.05).

h. To ask the subjects in both groups to write the same amount of out-of-class essays because 5 for the control group and 7 for the experimental group may make the treatment efforts different for the 2 groups, and thus distort the results.

i. To state whether subjects dropped out or not during the study process.



Comments: peng@indiana.edu
Dr. Peng's Home Page: Dr. Chao-Ying Joanne Peng
Copyright 1998, The Trustees of Indiana University