A Meta-Analysis of Reading
Recovery in United States Schools

A Meta-Analysis of Reading Recovery in United States Schools

D’Agostino, J. V., & Murphy. J. A. (2004). Educational Evaluation and Policy Analysis, 26(1), 23–38.

Background

The purpose of this study was to provide a more comprehensive evaluation of Reading Recovery in U.S. schools by using meta-analytic procedures. The meta-analysis allowed inclusion of many studies not considered in previous Reading Recovery review because of narrower guidelines. The study included two analyses: Analysis 1 included 36 studies, regardless of apparent quality, and Analysis 2 examined the 11 more rigorous studies that provided pretest and posttest scores from treatment and comparison groups. Results of the two separate analyses were compared to determine if study quality influenced the overall conclusions regarding the impact of Reading Recovery. By developing norm-referenced means and standard deviations for two distinct comparison groups, authors were able to assess discontinued and nondiscontinued Reading Recovery students’ test scores on multiple measures and at multiple points in time. (In Reading Recovery, the discontinued category means students students who have successfully completed lessons.)

Conclusions and Recommendations

The researchers found positive program effects for both discontinued and not-discontinued students on outcomes tailored to the program and outcomes on standardized achievement measures. Effects among discontinued children (those who successfully completed lessons) were greater. Researchers did not find large discrepancies in results between the less and more selective analyses; results on more rigorously designed studies seemed to converge with the bulk of available evidence. They found no evidence that methodological flaws or weaknesses in individual studies were responsible for previously identified effects. Analyses of follow-up studies showed that when compared to similar needy students, discontinued children widened the gap from posttest to second grade on standardized measures, and they closed the gap with average students.

Comments

Results of this meta-analysis indicated a lasting program effect, at least by the end of Grade 2, on broad reading skills. Contrary to conventional belief, the researchers found no evidence that prior observed effects could be explained completely by factors resulting from methodological flaws (e.g., regression artifacts).

This abstract first appeared in Schmitt, M. C., Askew, B. J., Fountas, I. C., Lyons, C. A., & Pinnell, G. S. (2005). Changing Futures: The Influence of Reading Recovery in the United States. Worthington, OH: Reading Recovery Council of North America.