A new working paper shows that declines in student achievement were significantly larger for students in schools that faced longer closures. Copyright: Shutterstock
Schools throughout the world closed in response to COVID-19 beginning in 2020. In a few cases, they quickly reopened, but in some countries, they remained closed far longer.
Country studies show large learning losses, which could cost this generation of students trillions of dollars in lifetime earnings. Global studies also document significant learning losses. International grade 4 reading scores declined an average of 33 percent of a standard deviation, equivalent to losing more than a year of schooling. Distance learning during the school closures does not seem to have helped very much.
PISA results show widespread learning losses
Last month the OECD released results from PISA, an international assessment representing the achievement of more than 156 million 15-year-old students. By combining data from previous assessment rounds and the post-COVID round for countries that have data for both time periods, we can see the evolution of test scores before and after COVID-19. We model the effect of closures on achievement by predicting the deviation of the most recent results from a linear trend in achievement.
This latest analysis in our new working paper reveals that school closures led to student learning losses and that those losses are widespread. We found that scores declined from 2018 to 2022 by an average of 14 percent of a standard deviation, roughly equal to seven months of learning. Losses were greater for students in schools that faced relatively longer closures, boys, immigrants, and disadvantaged students. Educational losses may translate into significant national income losses over time. These assessments represent 175 million 15-year-old students in 72 countries.
Departures from the time trend separately for each year of PISA assessment (95% confidence intervals)
Longer closures led to worse learning outcomes
What became clear is that the decline in achievement was significantly larger for students in schools that faced longer closures. Our analysis reveals that school closures led to student learning losses when using our methodology to account for differences due to pandemic school closures.
In contrast, countries with the shortest closures experienced relatively small losses. Ten percent of a standard deviation, or equivalent to 5 months of learning. Countries with average length of school closures experienced losses of around 13 percent of a standard deviation, or 7 months of learning. Countries with the longest closures experienced losses of around 20 percent of a standard deviation, or up to 12 months of learning.
Depending on the closure length, each week of school closures was associated with 0.15-0.17 points or around 0.2 percent of a standard deviation of additional decline in achievement for boys. At around 40 weeks of full school closures, the overall learning loss for boys is the same as for girls, but learning losses continue to increase for boys for longer closures. Students with an immigrant background lost 0.33 points or 0.4 percent of a standard deviation with every week of full school closures. In countries with shorter closures their learning loss was smaller, but for countries with around 25 weeks of closures, their learning loss was similar to native-born students, and it increases for longer closures.
Additional analyses reveal significant differences in the learning losses between students at different achievement levels, but also across countries that vary in the length of school closures. For countries with the average length of school closures the learning loss is similar for low-, average-, and high-achieving students. However, in countries with relatively short closures, the best students lost very little in terms of achievement. The decline is mostly observed among the average- and low-achieving students. In countries with longer closures, the learning loss is larger for the best students. In countries with the longest closures, the low-achieving students lost around 16-17 points (or 20 percent of a standard deviation), while those at the top of achievement distribution lost 25 points or more (29 percent of a standard deviation).
Learning loss estimates depending on student achievement quantiles and the length of closures
These differences in learning loss at different achievement levels in countries with short and long closures can be associated with differences in the overall achievement in these countries. Correlating average achievement with the length of school closures reveals that countries with the longest closures are also countries with the lowest achievement in PISA, while countries at the top of the PISA rankings closed schools for much shorter periods, on average. There are larger losses among the lowest achieving students in countries with short closures and high achievement, and larger losses among the highest achieving students in countries with long closures and low achievement.
Long-term consequences of school closures
COVID-19 school closures led to large learning losses, which will have economic consequences. The loss of human capital will affect this generation’s school children when they enter the labor market. Their future earnings will be lower than they would have been in the absence of the school closures. Countries will suffer because of the lower human capital and reductions in productivity. All this will constrain growth and development efforts.
To receive weekly articles, sign-up here
Join the Conversation