Educational technology (EdTech) is increasingly promoted as a solution to strengthen teaching and student learning, especially after COVID-19 which, in many countries, led to a boom in the use of technology in education. Is EdTech able to support improvements in student learning, or is it just hype?
In our study Using Education Technology to Improve K-12 Student Learning in East Asia Pacific: Promises and Limitations, we reviewed all rigorously published impact evaluations of education interventions in the East Asia Pacific (EAP) region that measured student learning outcomes to try to answer that question. We analyzed 52 evaluations in seven EAP countries which measured student learning and found that the 25 evaluations that focused on EdTech interventions had far more variation in impact than the non-tech interventions, with a range in standard deviation from –0.07 to 0.98, which means that some of the Ed-Tech interventions had a very large positive impact on student learning, while others reduced student learning.
Figure 1: Comparison of effect sizes (adjusted) with confidence intervals for EdTech, Has Tech, and Non-Tech education interventions in the EAP region
Why is there so much variation?
The three main factors that affect whether EdTech led to more or less student learning were (1) whether the intervention was implemented at scale or not, (2) who implemented the interventions, and (3) uptake and dosage.
- Size does matter
Most EdTech evaluations in our dataset focus on educational interventions that are pilots or very small scale. We along with others find that the effect size tends to diminish as the intervention scale increases. Consequently, most current EdTech evaluations in EAP are best viewed as tests of efficacy rather than effectiveness. Since evaluations commonly focus on small-scale pilot interventions, they tend to offer only limited insights into the broader impacts, costs, and capacities required when implementing on a regional or national scale. Moving forward, more evaluations should be conducted for EdTech interventions at scale.
- The implementer matters
Of the education studies we reviewed with the largest impact, 8 out of 11 were implemented by NGOs. This is likely because NGO implementers often directly monitor program progress, possess advanced skills, and have a heightened motivation for the success of their interventions. When government implementation was compared with NGO implementation for one of the studies, the positive impact on learning disappeared.
- Uptake, dosage, and participant characteristics
Our research indicates that average effect sizes often conceal the true extent of student and teacher participation in the intervention, including how much time they actually spent using the program or device (dosage), or whether they used it at all (uptake). Focusing on the average effect can obscure the diverse impacts on students with different characteristics, such as gender, income, and baseline learning levels.
Where do we go from here?
We found multiple large-scale studies with large student learning impacts using the “dual-teacher” model of remote instruction, including in Ghana, India, Pakistan, Mexico, China, and Uruguay. In a key study in rural China, remote instruction was scaled up to thousands of schools, reaching more than 100 million students during the evaluation. Despite this large scale, there was still a significant positive impact on student learning (0.23 SDs).
According to UNICEF, there are approximately 43 million children with disabilities in the EAP region. Our research unveiled limited evidence on how assistive technologies could enhance their learning outcomes. There is a pressing need to create, assess, and expand assistive EdTech programs tailored to meet the needs of all children.
In a nutshell, EdTech has lots of potential in East Asia Pacific, but it’s not a one-size-fits-all solution. As educators, policymakers, and tech enthusiasts, we need continuous research and collaboration to ensure that technology truly elevates learning for all students. This includes AI, which offers the potential of high-quality personalized learning experiences and lightening the workload for teachers. We need more in-depth studies to figure out the best way to integrate AI-driven EdTech programs into schools.
Policymakers and educational researchers can utilize these findings as foundational insights to explore effective EdTech programs, and refine their EdTech practices based on our recommendations, with the aim of effectively using EdTech to truly elevate learning for all students in East Asian and Pacific middle-income countries in the post-COVID-19 era.
To receive weekly articles, sign-up here
Join the Conversation