Impact Evaluation
In the past few weeks, the COVID-19 pandemic has shuttered schools in more than 160 countries, leaving more than 1.5 billion children – and 75% of enrolled students – out of school. This has sent many school systems scrambling to prepare materials for remote learning and to adopt new technologies to reach children in their homes.They are largely doing this without much of an evidence base to guide their decisions. Most technologies for remote learning have been tested under ideal conditions, often with small samples of children who have access to a computer and a well-trained instructor and who do not find themselves in the middle of a worldwide health emergency.
While we’ve never witnessed this kind of global aggregate shock to education before, using technology to deliver education services is not new. Television can reach children in their homes. Since the 1960s, the Mexican Telesecundaria program has provided remote learning through satellite television in multi-grade schools and in junior secondary education, targeting small remote villages that even today are unlikely to have high speed and consistent access to the internet. Millions of learners use it today in different countries (Costa Rica, El Salvador, Guatemala, Honduras and Panama, and even the United States and Canada), and a recent evaluation estimates high impacts on both subsequent educational attainment and income later in life.
Radio programming can also achieve broad coverage and help children in low-resource environments learn. Back in the 1920s, the British Broadcasting Corporation (BBC) in the UK began using radio to broadcast education programs. A more deliberate approach to delivering education services is interactive radio instruction that asks learners to react to questions and exercises through verbal response. In Nicaragua in the late 1970s, in one of the first randomized control trials funded by the World Bank, this kind of radio-based math instruction was found to increase test scores significantly (by 1.5 standard deviations).
There is huge heterogeneity between countries in technological infrastructure and institutional capacity to take advantage of online platforms that could be repurposed for delivering education services during a lockdown. An interesting example is Egypt, where they have recently developed a large repository of study materials in English and Arabic for students (kindergarten to secondary school) among other national platforms and tools to help teachers communicate with students online.
Will these strategies work in countries around the world to reach the school children who now find themselves at home? Evidence generated in less fraught times suggests that designers of technology-based interventions need to account for multiple dimensions of the context – quality of infrastructure, skills of educators, and frequency of use - in order to translate the use of digital technology into real learning. Thus, the optimal technology and optimal deployment strategy are likely to vary across countries.
When it comes to thinking about health – specifically, preventing and treating cases of Covid-19 – most people would demand that any policies and measures adopted by governments be based on solid evidence rather than intuition or ideology. This is why there are already ongoing randomized control trials to test new vaccines and treatments for COVID-19 before they are rolled out to the population at large. We would not abandon the scientific method for the sake of expediency since getting things wrong comes at a huge cost in terms of both disease and lives lost.
We should expect the same for our responses in education since getting things wrong could lead to permanent decreases in human capital across multiple generations.
This is why the Strategic Impact Evaluation Fund (SIEF) has added a COVID-19 emergency window to its recent call for proposals, which focuses on generating evidence on the role of technology in accelerating learning and skills among children and adult learners. SIEF funds randomized control trials and quasi-experimental evaluations. We are prepared to expedite the application process for the COVID-19 window to generate results that countries can use immediately as they deal with school closures and prepare for the reopening of schools, including with interventions to offset any losses in learning.
Are there any researchers prepared to act quickly and start to identify (i) remote learning that achieves high coverage, high take-up, and learning, (ii) supplementary interventions that might be needed to maintain children’s social and emotional well-being, and (iii) and successful strategies for transitioning children back into school? If so, please apply. Quickly. We are in the midst of a global education emergency where evidence-base approaches are urgently needed.
Join the Conversation