Assessment of the School District of Lancaster’s Early Reading First Program
In 2008, the School District of Lancaster was awarded a four-year U.S. Department of Education Early Reading First (ERF) Grant. As the evaluator of record for the District, the Center had primary responsibility for evaluating the impact of the project. We worked with the District’s team to refine the ERF evaluation plan, develop data collection tools and reporting methods, collect and analyze data, and report results to the District and the Department of Education. In addition, the Center participated on the ERF Advisory Committee to help set policy, analyze data, evaluate recent activities and accomplishments, and suggest program improvements.
The primary goals of the ERF program were to improve prekindergarten children’s oral language, phonological awareness, print awareness, and alphabet knowledge through intentional, age-appropriate language and literacy experiences. Federal grant monies funded the program for four years, from September 1, 2008 through August 31, 2012. Originally implemented in 14 of the District’s 21 prekindergarten classrooms, ERF was later expanded to include all 21 classrooms.
Working closely with the ERF Leadership Team, the Center helped create a detailed evaluation plan to measure the overall success of the program, including several specific measures as required by the Government Performance and Results Act of 1993 (GPRA). The Center’s responsibilities under the plan were to 1) evaluate the process of implementing the program, and 2) assess the impact of the program on the learning and achievement of the participants.
The Center collected data from the District, including student test scores and teacher observation results, each fall and spring. Working with the ERF Project Manager and preschool teachers, the Center developed new ways for them to efficiently collect and aggregate student test scores and other relevant data. As the District’s program evaluator, the Center maintains a database of District student information including academic achievement, attendance, behavior, demographics, and program participation. Demographic information about all ERF students was therefore readily available. In addition to gathering this quantitative data, the Center and ERF staff collected qualitative data about the program.
Each year the Center assessed student achievement and improvement in the following areas: Phonological awareness, receptive vocabulary, rhyming, alliteration, and picture naming. We also analyzed student test results by demographic subgroup, including race, ELL status, gender, IEP status, school cluster, and grade level. Finally, the Center examined the data to assess results based on the GPRA measures: Oral language skills, letter recognition, cost per preschool-aged child, and teacher performance. Because ERF was a multi-year program, the data were analyzed not only within each year, but across years as well, to gauge change.
Also each year, the Center produced an extensive ERF evaluation report. These reports summarized research methodologies, student and teacher assessments, demographic analyses, and student progress each school year from 2008 to 2011, comparing results over the years where applicable. A final 2012 report provided a comprehensive view of the program’s successes during its four years, and discussed the long-term impacts of the program. The Center’s analysis of the data collected showed that the program met or exceeded all of its federal performance measure targets each year. In addition, the data showed a long-term positive impact: by the time students reached kindergarten and first grade, those who had participated in ERF were significantly more likely than their peers who had not participated to score at or above benchmark levels.
Using the Center’s analysis and reporting, the District was able to demonstrate the success of the Early Reading First program. The data and results were included in the District’s annual reports to the Department of Education, and throughout the four-year funding period, the District used the Center’s data analyses to inform decisions about the project and create program improvements.