Practice Areas

Randomized Controlled Trials (RCTs)


CEEP has a strong history and background conducting randomized controlled trials (RCT). CEEP staff have published and presented on issues related to scientifically-based research and experimental design evaluations. Recent RCT evaluations include the following selected projects.

Experimental Design Evaluation of Full-Day Kindergarten

Client: U. S. Department of Education
(Description: Show/Hide)

CEEP provided rigorous evidence related to both the short- and long-term impact of full-day kindergarten (FDK) on students’ academic outcomes. Kindergarteners in three different elementary schools within two different Indiana school districts participated in the study. Students applying for FDK programs in each school were randomly assigned to either the full-day (treatment) or half-day (control) program. Student achievement was recorded via performance on the Stanford 10 Achievement Test and the Curriculum-Based Measurements to benchmark all students. Additionally, the following non-achievement data was also collected as part of the study: attendance data; special education referral rates; grade retention rates; data regarding time usage in full- and half-day kindergarten classes; a parent/guardian questionnaire regarding kindergarten students’ outside-of-school activities (e.g., extracurricular activities, before- or afterschool care, tutoring); and a teacher focus group.

Evaluation of the CALM Program for High School Chemistry Students

Client: U. S. Department of Education/Institute of Education Sciences (IES)
(Description: Show/Hide)

CEEP has begun a five-year IES evaluation testing the efficacy of the Computer Assisted Learning Method (CALM) on student learning and achievement in high school chemistry. CEEP is using a randomized controlled trial design to analyze the impact of CALM on student learning, long-term retention of knowledge, and participation rates in additional science courses. CEEP is also analyzing the differential effects on CALM based on gender, ethnicity, and socio-economic status. CALM is a fully developed intervention for chemistry education, currently providing an online learning tool to high schools throughout Indiana. Despite its frequent use, the high interest expressed in the intervention by other Indiana schools and nationwide, and the anecdotal and qualitative data that supports the effectiveness of CALM, to date no strong evidence has been collected related to the impact on student outcomes. This study provides an opportunity to collect the types of empirical evidence needed to further inform program and policy decisions, and to implement the strong methodology needed to provide scientific evidence of the efficacy of CALM.

Technical Assistance to Teaching American History Program Using Experimental and Quasi-Experimental Design Evaluation

Client: U. S. Department of Education, through subcontract with Synergy Enterprises, Inc.
(Description: Show/Hide)

For two years, CEEP provided evaluation technical assistance (TA) to OII’s Teaching American History program in an effort to increase the program’s capacity and quality of scientifically-based evaluation designs. Specific areas of TA included the following: individualized reviews and recommendations for strengthening evaluation designs; teleconferences and/or individualized consultation via phone or e-mail with grantees to resolve methodological issues; training and education provided at the national TAH directors’ meeting; and teleconferences, educational forums and online resources on “hot topics” to increase grantee’s internal evaluation capacity. As part of this contract, CEEP developed rubrics for analyzing strengths and weaknesses of evaluation designs; created online TA logs; and developed web-based resources available to grantees.

Voyager Passport in 21st CCLC

Client: SEDL
(Description: Show/Hide)

In collaboration with the Kentucky Department of Education, CEEP conducted a rigorous longitudinal evaluation of the effects of participation in an afterschool program using a randomized controlled trial design. This planned variation study design compared the impact of previously established 21st Century Community Learning Center (CCLC) programs (control) with the impact of CCLC programs that included the Voyager Passport reading program (Passport). The study tested the hypothesis that economically disadvantaged youth in grades 2-5 who participated in a high-quality afterschool program with Passport achieved significantly greater learning gains over a two-year period than disadvantaged youth who participated in the same afterschool program without Passport. The study also analyzed the impact of Passport on general student academic outcomes, its impact on special education referrals and placement, grade level retention, regular school day attendance, and participation in afterschool programs themselves. Data collection methods included: multiple administrations of the Dynamic Indicators of Basic Early Literacy Skills, the Stanford Achievement Test, 10th Edition, periodic observations of program implementation using a structured rubric developed by Passport, as well as self-report survey data from teachers and other key stakeholders.