We also considered the evidence framework used in the Every Student Succeeds Act (ESSA), which outlines four tiers of evidence depending on the certainty that a program will create positive child or youth outcomes based on the strength of the research design. However, due to the variety of outcomes studied, limitations of the available evidence base, and variability regarding what constitutes a small or large effect size depending on measure and age level, 1 the committee concluded that we could not accurately classify the magnitude of the benefit for all outcomes beyond three categories: positive, zero, and negative. Preventive Services Task Force framework particularly appealing, as it encompasses two important dimensions: certainty of benefit (with high, moderate, and low levels of certainty) and magnitude of net impact (substantial, moderate, small, zero, and negative). The committee considered a number of classification and rating schemes.
In deciding how to classify research evidence of program effectiveness, the committee was guided by a desire to be helpful to policy makers and practitioners looking to adopt evidence-based practices and to use a scheme that would fit the available evidence. Much of that discussion is descriptive, but as we turn to examining program effectiveness it is important to determine how confident we are that outcomes measured by the program are the result of the program itself rather than other factors, such as the passage of time or pre-existing differences between participants and those who did not participate. The earlier chapters drew on a range of research to describe what children and youth do during the summer and how summer affects their development.
CLASSIFICATION OF PROGRAM EFFECTIVENESS EVIDENCE