The 2001 NAEP in U.S. History
by John J. Patrick
June 2002


Since its inception by the United States Congress in 1969, the National Assessment of Educational Progress (NAEP) has collected information about what students in the United States know and can do in core subjects of the school curriculum.  NAEP is administered by the National Center for Education Statistics (NCES) in the United States
Department of Education's Office of Educational Research and Improvement (OERI). NAEPs in United States history were conducted in 1986, 1988, 1994, and 2001.  This Digest discusses (1) the framework of the 2001 NAEP in
U.S. history, (2) the assessment procedures, and (3) the findings of this national assessment of achievement in U.S. history by students at grades 4, 8, and 12. 

THE FRAMEWORK

The framework developed for the 1994 NAEP in U.S. history was used again in 2001.  Thus the findings of the 1994 assessment can be compared with those of the 2001 NAEP in U.S. history.  A 22-member planning committee composed of historians, teachers, and history educators took primary responsibility for developing a framework to guide the structure
and content of the 1994 NAEP in U.S. history.  In addition, several hundred persons -- historians, history educators, school administrators, representatives of professional associations, and members of the general public -- contributed to the framework development by participating in public hearings or writing critical reviews of drafts of the framework document.  This process yielded the "U.S. History Framework for the 1994 National Assessment of Educational Progress," which set specifications for
the subsequent development and administration of the background questions and test items that constitute the assessment.  This framework also provided the structure for interpretation of the assessment results.  

The core of the framework consists of four themes in U.S. history:

     *  Change and Continuity in American Democracy: Ideas, Institutions, Practices, and Controversies
     *  The Gathering and Interactions of Peoples, Cultures, and Ideas
     *  Economic and Technological Changes and Their Relation to Society, Ideas, and the Environment
     *  The Changing Role of America in the World

The four themes of the framework relate to each of the following  eight periods of history:

     *  Three Worlds and Their Meeting in the Americas (Beginnings to1607)
     *  Colonization, Settlement, and Communities (1607 to 1763)
     *  The Revolution and the New Nation (1763 to 1815)
     *  Expansion and Reform (1801 to 1861)
     *  Crisis of the Union: Civil War and Reconstruction (1850 to 1877)
     *  The Development of Modern America (1865 to 1920)
     *  Modern America and the World Wars (1914 to 1945)
     *  Contemporary America (1945 to Present)

The framework specifies ways of knowing and thinking about U.S. history.  Two cognitive levels were considered when developing the exercises for the national assessment: (1) a lower cognitive level involving recall and comprehension of knowledge and recognition of perspectives of different persons and groups in history and (2) a higher cognitive level involving analysis and interpretation of issues and events and use of evidence to make warranted generalizations about the past. 

ASSESSMENT PROCEDURES

Common test items of the 1994 and 2001 assessments in U.S. history measured the knowledge and cognitive skills components of the framework. The assessment included both multiple choice items and constructed-response items, open-ended questions that challenge students to use information and ideas to express in writing their thoughts on
sources and events in history.  In line with the framework, most of the assessment time involved exercises that required students to use higher-level thinking skills. 

Comparable procedures were used to sample and assess the student population in 1994 and 2001.  The 2001 NAEP, like the 1994 national assessment, was administered to representative national samples of both public and nonpublic school students.  Approximately 29,000 students were sampled: 7,000 4th graders, 11,000 8th graders, and 11,000 12th graders.
The national sample consisted of 1,108 schools: 365 at grade 4, 369 at grade 8, and 374 at grade 12. 

Students responded to U.S. history assessment items and questionnaires about personal characteristics and experiences possibly related to achievement in U.S. history.  Teachers of the student respondents completed questionnaires about curricular content and classroom practices. 

FINDINGS

Results for each grade -- fourth, eighth, and twelfth -- are reported according to three achievement levels: Basic, Proficient, and Advanced. The Basic level indicates partial mastery of knowledge and skills that are prerequisites for competency in U.S. history.  The Proficient level signifies competent academic performance in the knowledge and skills of U.S. history.  The Advanced level designates superior performance in U.S. history.  The particular knowledge and skills denoted by each achievement level were determined and the distinctions between achievement levels were set by broadly representative panels of experts (e.g., master teachers, historians, education specialists, and members of the general public). Thus, the achievement levels represent collective judgments about what students should know and be able to do in U.S. history at grades 4, 8, and
12. 

Sixty-seven percent of fourth graders, 64 percent of eighth graders, and 43 percent of twelfth graders attained the Basic level.  Eighteen percent of fourth graders, 17 percent of eighth graders, and 11 percent of twelfth graders achieved the Proficient level.  Two percent of fourth graders, 2 percent of eighth graders, and 1 percent of twelfth graders
reached the Advanced level.  The achievement of 4th and 8th graders rose slightly between 2001 and 1994.  At 12th grade average scores and achievement of students stayed the same in 2001 as in 1994. 

In general, the results of the 2001 NAEP in U.S. history are as disappointing as those of 1994.  A striking indicator of disappointing performance is the high percentage of students in grades 4, 8, and 12 who did not reach even the Basic level of achievement.  Thirty-three percent of fourth graders, 36 percent of eighth graders, and 57 percent of twelfth graders were at the Below Basic level.   

Performances on both the 1994 and 2001 NAEP in U.S. history varied significantly by certain group memberships.  At all grades, for example, students identified as white and Asian/Pacific Islander tended to score higher than did those identified as black or Hispanic; however, the gap in achievement between white and black 4th graders was smaller in the 2001 NAEP than it was in the 1994 assessment.  And the gap between white and Hispanic 12th graders narrowed from the 1994 to the 2001 NAEP in U.S. history. 

In 2001 as in 1994, students in nonpublic schools tended to perform better than did students in public schools.  And students in Catholic schools scored higher than students in other nonpublic schools and students in public schools. 

At all three grade levels, students who qualified for the federally funded free or reduced-price school lunch program tended to score lower on this assessment than students ineligible for this program.  This finding suggests there may be a general relationship between lower socioeconomic status and lower achievement in U.S. history. 

Certain instructional activities and classroom experiences were related to achievement in U.S. history as measured by the 2001 national assessment.  For example, a negative general relationship existed between daily general use of computers in social studies or history classes and student achievement; however, relatively few students reported using a computer in social studies or history classes. A positive relationship emerged when 8th and 12th graders used computers for specific instructional activities such as conducting research and writing reports on topics in U.S. history. 

There was a positive relationship between more time spent in 4th grade classrooms on the teaching and learning of history and student achievement.  Further, 4th graders who spent more time reading and learning from a textbook achieved higher assessment scores than those who spent less time in this kind of instructional activity. 

Eighth graders who used primary source materials weekly earned higher average national assessment scores than those who experienced less or no use of this kind of instructional material.  Twelfth graders who read biographies and other kinds of stories in history performed better than those who never experienced this kind of classroom assignment. 

SOURCES OF INFORMATION ABOUT THE NAEP IN U.S. HISTORY

Information in this Digest is from "The Nation's Report Card: U.S. History 2001."  To order this publication or any other NAEP-related products, contact Education Publications Center (ED Pubs), U.S. Department of Education, P.O. Box 1398, Jessup, MD 20794-1398; toll-free 877-433-7827; FAX 301-470-1244. This publication is also available from
the ERIC Document Reproduction Service (EDRS), 7420 Fullerton Road, Suite 110, Springfield, VA 22153-2852; toll-free 800-443-3742.   

The NAEP Web site contains information about the NAEP in U.S. history and general information about assessment, publications, and analysis tools used by various NAEP projects  <http://nces.ed.gov/nationsreportcard>. 


John J. Patrick is Director of the Social Studies Development Center and Professor of Education at Indiana University.  He was a member of the Planning Committees for the 1994 and 2001 NAEP U.S. History Projects.  He is a member of the NAEP U.S. History Standing Committee.