|
 |
Unit 2: Writing (or Reading) a Research Paper
Introduction
Lecture slides
Instructor notes
Readings
Learning activities
Web resources
Writing (or reading) a Research Paper
This unit may be thought of as four separate topics:
Most of the published articles we will read this semester are organized as follow:
- Introduction, which may contain some of the following:
- Problem statement. This is often phrased as a question or as a dramatic statement in order to capture the reader's attention. Then a dilemma is elucidated, which of course, the present research study will hope to resolve. Here is an example from Rose, Chassin, Presson, and Sherman (1996)
Cigarette smoking is the single largest cause of premature and avoidable death and disability in the United States (U. S. Surgeon General, 1989). Although rates of adult smoking have been declining since the publication of the 1964 Surgeon General's Report, epidemological data suggest that these successes have not uniformly been distributed among the population.
- Objective. Sometimes the introduction omits the problem statement and instead presents a simple statement indicating what the investigators hope to achieve. They may state their purpose and suggest potential benefits that may derive from this research. Some journals, such as the British Journal of Educational Psychology and American medical journals such as Pediatrics place subheadings -- such as, objective, design, methods, results, and conclusion -- in the abstract so that reader can spot them easily.
When you are thinking about writing a research paper, ask yourself, What, exactly, do I wish to find out? Is it a researchable problem? What do I need to know in order to conduct this research -- What kinds of knowledge? Is the data available? How much time will it take? What kind of monetary outlay is needed?
- Theoretical statements. Research conducted without a guiding theory, though interesting and maybe even fun, is aimless. The phrase "dustbowl empiricism" applies to empirical investigations that are not guided by theory. Theories serve several purposes:
- Integrate the findings of past empirical research,
- Reconcile conflicting findings, and
- Suggest areas for future research.
When writing a paper, you will want to ask yourself, To which theory or conceptual framework can my problem be linked? After identifying a theory or conceptual framework ask yourself, What are the criticisms of this theory or conceptual framework? In what ways might this particular theory or framework limit the investigatory methods I can use to conduct an empirical study? As always, try to be aware of the researcher's (and your own) usually tacit assumptions.
- Literature review. For virtually any problem you wish to investigate someone has published a related article. Your task is to determine the how much of the history of the problem your reader needs to know, what (competing) theories/frameworks exist that are relevant to this problem, what empirical facts are generally accepted as being true, and anything else the literature might contain that you think readers should know about the problem you have selected.
- Variables. The introductory section usually describes variables in abstract, conceptual terms. These variables are the components or attributes of a theory. Consider Stankov and Roberts' (1998) conceptual definiton:
Emotional intelligence has been defined as "the ability [italics added] to monitor one's own and others' emotions, to discriminate among them, and to used the information to guide one's thinking and actions" (Salovey & Mayer, 1990, p. 189). A number of researchers thus view the capacity to process affective information as a "mental ability" or "aptitude" in the conventional sense.
Terms such as "emotional intelligence," "mental ability," "aptitude," "emotions," and "thinking" are all abstractions that must be defined operationally before they can be investigated. Note that although the term "variable" may not be mentioned, nonetheless your task as a reader is to identify those attributes that can take on a variety of values. At some point in an empirical research article, the authors should state the exact procedures they will use to measure these abstractions. Also, if the study is experimental, somewhere in the introduction the authors should identify the independent and dependent variables. Other variables that may influence the dependent variable should be mentioned and controlled.
- Hypothesis. The hypothesis is simply a more concrete and specific statement of a more generally stated topic. The hypothesis should be stated in such a manner that it can be tested. For example, while a problem statement might be phrased as: Does increased spending on education produce increased student performance?, a hypothesis statement might read: No relationship exists between per pupil expenditure and pupil performance in grades 3 through 6 on the Iowa Test of Basic Skills.
Later we will distinguish between a research hypothesis, which is like the natural language statement above, and a statistical hypothesis, which might be phrased "The correlation between per pupil expenditure and test performance is zero."
- Methods. This section usually contain the following
- Operational definitions. If student performance is the dependent variable in a study, the researchers must specifiy how it will be measured. Some possibilities include performance on a well-know test such as the Iowa Test of Basic Skills, a state level test such as Istep, a test developed by the school district, or a classroom test.
The researchers may specify some other kind of perfomance assessment that does not require paper and pencil. For example, suppose researchers wanted to determine whether a "new and snazzy" method for teaching directional navagition to third graders was superior to the "same old stuff" approach. The "test" might consist of dropping students trained with both methods into a remote area of the Hoosier National Forest and seeing how quickly each reaches a specified point.
Independent variable(s), likewise, must be operationally defined.
- Measurement and Instrumentation. The measurement of cognitive and affective variables is often one of the weakest parts of research studies. If possible, you should select an instrument that already exists. The authors of an existing instrument should report where the reader can find information about the pyschometric properties (i.e., the reliability and validity) of the instrument. If the researchers are using an instrument they specifically constructed for this research, they should report the instrument's reliability and validity.
In addition, when reading an article, always keep in mind the level of aggregation. If researchers report that the "new and snazzy" method is superior to the "same old stuff," do they base that claim on the performance of individual students? Classrooms of students? Other groupings of students?
- Research design. What is the author's stated goal? Is it to describe a phenomenon? Do the authors' wish to determine whether or not a relationship exists between and/or among variables? Do they wish to be able to state that a causal relationship exists between variables?
Description. If the authors' goal is description, then the design will have few controls and (should) make no definitive statements about relationships among variables. Articles that report the results of surveys often do nothing more than describe.
Relationship. The authors may state they wish, for example, to determine if a relationship exists between, say, the lexical difficulty of school textbooks and student verbal performance. We would expect such researchers to specify operationally how they plan to measure "lexical difficulty" and "verbal performance." They would need to find some textbooks that are less difficult than other textbooks and see if students who used the easier textbooks also exhibited lower verbal performance. The researchers should not, however, make any statements about causation, for they have not controlled other variables that may contribute to verbal performance. Such designs are often referred to as correlational.
Experiment. If the authors wish to make a causal statement (e.g., "lexically simple textbooks used over several grades contributes to lower verbal performance"), the researchers must conduct an experiment. They cannot just measure lexical difficulty of texts and student verbal performance. Instead, they must be certain that groups of students are equivalent in all relevant aspects and only then expose one group of students to lexically simple texts and the other group to lexically difficult texts. If, after a period of time, the students who use lexically difficult texts show consistently higher verbal performance, then the researchers might be justified in making a causal statement.
To summarize: The three major categories of research designs are (a) descriptive (includes much survey research and all qualitative research), (b) correlational (look for relationship between variables), and (c) experimental (the researchers control all variables that may impact the dependent variable and allow only the independent variable to vary). Later, we will add ex post facto and causal-comparative to this discussion.
- Sampling. Random sampling and random assignment of subjects to groups is generally considered the best way to insure that groups are equivalent at the beginning of an investigation. The authors should state specifically how they selected their sample and provide whatever evidence they can that the sample is representative of the population to which they wish to generalize. When you think about conducting research, some of the critical questions include how you identify the population, how you will draw a random sample, and how you will randomly assign subjects to groups.
- Data collection. How many times were the data collected? If the study is cross-sectional, the data likely were collected once. If the study is longitudinal, the data were collected repeatly. From how many different groups were the data collected? Was more than one instrument used? If so, how does each relate to the other instruments? If the study involves interviewers, do the authors describe how they were trained? Do they report anything that might lead us tho think that interviewers were consistent across interviewees? If the study involves observers, say of classroom discussions, do the authors describe how the observer were trained? Do they report inter-observer reliability?
- Ethical considerations. Although it is rare that a research article contains "ethical considerations" as a separate heading, you must be aware of the subjects (aka "participants") and be sure the authors did not treat them in any way that might be considered unethical. You should explore the following web page that describes the procedures used at Indiana University to insure the ethical treatment of subjects: Bloomington Campus Committee for the Protection of Human Subjects.
- Data analysis. The authors should describe clearly the descriptive and inferential statistics they use and assure the reader that the assumptions required for the inferential procedures were checked and that the data do indeed conform to the assumptions.
- Results.
- Descriptive statistics. The authors may provide tables (and maybe graphs) that describe the performance of the groups on the dependent variables. In addition, the authors may present statistics to convince the reader that the data conform to the assumptions required for performing statistical tests. Wilkinson (1999) presents several useful ideas.
- Inferential statistics. When appropriate, the authors should report not only the statistical test (e.g., z-test, t-test, F-test, etc.), but also the strength of the difference (e.g., the effect size).
- Conclusions.
This section may also contain interpretations, recommendations, suggestions, speculation. You should expect the authors to state whether or not their original hypothesis was supported. They also should discuss the implications of the present findings in relationship to the theory discussed in the introduction.
At this point in life each of you have sufficient professional and/or academic experience that certain ideas presumably interest you more than others do. You should identify the broad ideas that interest you, narrow one of them, and think about how this idea is related to student learning. This idea may be suitable for the term paper in this course.
Nonetheless, some students still experience difficulty in selecting a idea for the research paper. Keep in mind that the idea you select for the paper in this course is not as important as is the manner in which you develop and write about the idea. That said, a idea must be selected before it can be developed.
If you have already identified a research area that interests you, Mitchell (1999) lists 21 Ways of Generating Research Ideas from Previous Research:
- Find gaping omissions.
- Repeat studies (but modify some critical aspect).
- Do a study suggested by the journal article's author(s).
- Repeat the study with a different group of participants.
- Look for situational factors that may moderate the effect.
- Look for factors that were not controlled.
- Reduce the effects of expectancies.
- Use more realistic amounts of the treatment factor.
- Uncover the functional relationship.
- Use more realistic stimulus materials.
- See if another factor would have the same effect.
- Bridge fields and try to find a practical implication of the research.
- Look at the studies from a different level of analysis.
- Look for patterns in conflicting studies.
- Look for a factor's immediate relationship to other variables.
- Look at long term effects.
- Look for "down the road" effects.
- Repeat the study using a different measure of the same construct.
- Repeat the study with a more sensitive way of detecting the effect.
- Take advantage of "component" measures.
- Take advantage of measures of entirely new concepts.
Mitchell (1999) also lists Six Idea Generation Techniques Applicable to Common Sense, Theory, and Literature Searches:
- See if the results would generalize to different participants or settings.
- Look for moderating variables that would either strengthen, weaken, or reverse the observed/proposed relationship between the variables. Asking "When does the opposite occur?" may help you think of moderating variables.
- See if you can apply it to a practical problem.
- Reconcile contradictions between conflicting studies, theories, or cliches.
- See if you can more precisely state the relationship between the variables.
- Examine variables that may mediate the relationship. What is the physiological or cognitive mechanism that accounts for the relationship? Can we measure those mediating processes to see if they really do occur when the stimulus is introduced? Can we manipulate these processes and see if manipulating with these underlying processes affects the stimulus-response relationship?
Another potentially useful source of idea generation is: McGuire, William J. (1997). Creative hypothesis generating in psychology: Some useful heuristics. Annual Review of Psychology, 48, 1-30. Scanning other articles in secondary sources such as the Annual Reviews may also generate ideas for research.
After you select a general idea, think about writing a hypothesis. The hypothesis is a tentative assertion about a phenomenon. It is a statement to be tested. The hypothesis is a sentence that takes a stand, makes a point, states what variables might be related to other variables in a particular manner. When writing a research paper, you first write the hypothesis and then search for empirically based articles that might provide evidence relevant to your hypothesis. When we speak of evidence, this term include both articles that support your assertion as well as those that refute your assertion. Some examples of general ideas, which I refer to as topics, and hypothesis statements follow.
- Topic: Television watching by teenagers
- Hypothesis: Television watching weakens the academic performance of high school students.
- Topic: Television watching and violence
- Hypothesis: Limiting the amount of television watching by children reduces their frequency of violent behavior.
 Back to Top
The advent of computers and the internet has supplemented, but not fundamentally changed, the way we search for articles. Before computers one had little choice but to trudge to the nearest research library and pour through indices such as Psychological Abstracts, Eric, the paper-based card catalogue, and to look through the past journals stored on library shelves. These methods are still useful -- and recommended -- if you are in physical proximity to a research library. As libraries continue to augment their brick and mortor presence with information through a wire, searching by computer becomes more attractive.
As you all know, indices such as Psych Info (the on-line supplement to the paper-based Psych Abstracts), and Eric are available on-line. You can download and/or print references that appear to be relevant. Since 1996 many Eric documents that are not journal articles can be view on-line. During the past few years an increasing number of academic journals have permitted their contents to be accessed electronically. Some permit access by anyone from their own web page while others permit access only through a library or for a fee. Recently, new "journals" began to appear solely on the internet (e.g., Educational Policy Analysis Archives and Journal of Statistics Education).
The starting points for accessing this information at Indiana University is the library home page. Sometimes you may prefer to use one of the general purpose search engines, such as Google. The Learning Activities section contains some (optional) exercises for those of you who would like to try searching for articles via the internet.
Back to Top
In his book Style: Ten Lessons in Clarity and Grace, Joseph Williams writes:
The ability to write clear, crisp sentences that never go beyond twenty or so words is a considerable achievement. You'll never confuse your reader with sprawl, wordiness, or muddy abstraction. But if you never write a sentence longer than twenty words, you'll be like a pianist who uses only the middle octave: You can carry the tune but not with much richness or variation.
Every competent writer has to know how to write a concise sentence and how to edit a long one down to comprehensible length. But a writer also has to know how to manage a long sentence gracefully, how to make it as clear and as vigorous as a series of short ones.
Chapter 2 (37 pages) of the Publication Manual of the American Psychological Association is devoted to writing style and grammar. Although we do not have the time to concentrate on style and clarity and grace, you should be aware of the importance of written expression and edit your text and polish your prose before submitting your papers. The Learning Activities section has some links that may be informative.
Back to Top
Reserved for Lecture Slides
No lecture slides this time.
Back to Top
Press the button for notes.
No instructor notes this time
Back to Top
Required:
Fraenkel & Wallen, Chapter 2.
Fraenkel & Wallen, Chapter 5.
Robinson, Ann (1988). Thinking straight and writing that way: Publishing in Gifted Child Quarterly, Gifted Child Quarterly, 32, 367-369.
Robinson asks provocative questions such as "What's the Point?" "Is this author killing flies with an elephant gun?" and "Would George Orwell approve?"
Suggested:
Wilkinson, Leland (1999). Statistical Methods in Psychology Journals: Guidelines and Explanations. American Psychologist,Vol 54, No 8, 594-604.
-- or --
Wilkinson, Leland (1999). Statistical Methods in Psychology Journals: Guidelines and Explanations. American Psychologist, Vol 54, No 8, 594-604.
<
Back to Top
- Use http://www.google.com to find the url for the American Education Research Association and browse through the latest issue of Educational Researcher.
- The instructor at this site has started a piece about writing the quantitative paper. Garson. You may find his ideas useful even though navigation becomes tiresome.
- See if you can find the full text version of the following article through Eric. We will look at this piece later in the semester when we consider qualitative methodology. (http://www.iub.edu)
Judith C. Lapadat, Judith C., & Lindsay, Anne C. (1998). Examining Transcription:
A Theory-Laden Methodology. Paper presented at the Annual Meeting of the American Educational Research Association, San Diego, CA, April 13-17, 1998.
- Which libraries in the Big Ten have the following book? Hint: Go to the library page and look for CIC or "Big Ten Virtual Library." (http://www.iub.edu)
Gilovich, Thomas (1991). How we know what isn't so: The fallibility of human reason in everyday life. New York: Free Press.
If you are associated with Indiana University and have your network ID updated, you can checkout books via the internet from other Big Ten libraries if the title you need is not owned by the IU libraries.
- Use the full text links available through the Indiana University library to view the McGuire article (Heuristics for idea generation). (A starting point is http://www.iub.edu)
- Recently I read the following in a more or less popular magazine:
[A] ...bit of recent evidence comes from a study in Pediatrics, conducted by five family physicians in Pittsburgh and Illinois. The doctors compared tens of thousands of local pediatric records, stretching from 1979 to 1996, and found evidence of grave increases in "psychosocial problems" among children 4-15.
Your task is to tell me if I were able to see the full text of this article on the screen and if I were able to print the article.
First Critique: Due February 2
Read the following article and write a critique. Send your document to me as an e-mail attachment.
Rosenthal, Robert, & Jacobson, Lenore (1966). Teachers' expectancies: Determinants of pupils' IQ gain. Psychological Reports, 19, pp. 115-118.
This is a critique of the Helm & Burkett (1989) article. It was written by a student in a previous Y520 class and may be used as a model for writing the Rosenthal critique.
Back to Top
See the Virtual Library/Research Tools
Back to Top
Last Updated: 01/01/19
|
