Factors that help disadvantaged students beat the odds: Evidence from PISA

A January study by the Organization for Economic Cooperation and Development (OECD) examines academic “resilience”—the capacity of disadvantaged students to succeed despite adverse circumstances—in more than seventy education systems that participated in the 2006, 2009, 2012, and 2015 iterations of the Program for International Student Assessment (PISA). Authors sought to determine the role that school environment and academic resources play in resiliency, and examined differences in rates of resilience in participating countries over time.

The study defines “resilient students” as those in the lowest socioeconomic quartile in their own country who achieve at or above Level 3 (out of 6) on all of PISA’s three subjects—reading, math, and science. “Level 3 corresponds, in each subject, to the highest level achieved by at least 50 percent of students across OECD countries on average,” explain the authors. They characterized school environment using the OECD’s survey data on truancy—how many students skipped a whole day in the last two weeks—and the frequency of classroom disruptions—insubordination, excess noise, etc. And the study measured school resources with an index comprising three variables: the availability of computers per student, the number of extracurricular activities offered, and the average class size of each school.

The study’s main finding was that, after controlling for the socioeconomic status of schools’ student bodies, school environment and academic resources explain, on average, about one-third of the variation in the number of resilient students between schools in a given country. But the effect of these variables differed between nations. In all but nine countries, for example, school environment has a statistically significant, positively associated effect on resilience. But for extracurricular activities—a component of researchers’ academic resources index—that was only true in a dozen nations. Moreover, a school’s environment was far more significant than its resources. “Attending orderly classes in which students can focus and teachers provide well-paced instruction is beneficial for all students, but particularly so for the most vulnerable students,” concluded the authors. And “resilience is only weakly related to the amount of human and material resources.”

The study also found that several countries, including Germany, Israel, Japan, Norway, Poland, Portugal, Slovenia, and Spain, saw their proportion of resilient students increase over time. But it dropped in many other places, like Finland, South Korea, New Zealand, Austria, Canada, Hungary, Iceland, Sweden, and Slovakia. Based on the main finding, this likely has something to do with changes in environment and resources. But the authors also found that student- and school-level socioeconomic status has a significant effect on rates of resiliency (hence SES being controlled for), so fluctuations in countries’ economies probably also play a sizable role.

The study does, however, have some limitations. Most importantly, all of these data are based on answers given by students, teachers, and schools on questionnaires that OECD includes as part of its PISA administration. It also doesn’t look at student growth over time.

Nevertheless, with the ed reform community perpetually abuzz about achievement gaps and discipline policies, this study provides valuable insight into contributing factors. It suggests that less disruptive classrooms may boost disadvantaged students’ achievement, and provides some evidence that school resources can help too.

SOURCE: Agasisti, T., et al. "Academic resilience: What schools and countries do to help disadvantaged students succeed in PISA", OECD Education Working Papers. (January 2018).