Digital personalized learning: A cost-effective solution for math remediation for college students in Ecuador

Many students who graduate from high school are academically unprepared for college, especially in subjects like mathematics. Globally, data reveals that 40 to 60 percent of lower secondary education graduates struggle with mathematics proficiency  (Figure 1). Consequently, many college-entrants grapple with math skills gaps, hindering their academic progress. This lack of academic readiness is one of the main reasons why about 40% of college enrollees fail to  obtain a degree. The COVID-19 pandemic worsened this situation, exacerbating learning losses on core foundational skills, especially among socioeconomically disadvantaged students.

Figure 1: Percentage of students at the end of lower secondary education achieving minimum proficiency in mathematics.

Source: UNESCO Institutes of Statistics (UIS), 2019.

Remedial programs aimed at addressing these issues are generally costly and difficult to implement. Students with skills gaps often lose motivation and struggle to keep up with their coursework, especially in information technology and communication (ITC) and engineering programs. Universities worldwide implement in-person remedial programs, which research suggests can help students progress into their second year. For instance, a study in Chile found that students who participated in counseling and math remedial courses achieved better academic results than their counterparts who did not. However, these programs rely heavily on tutoring, making them costly and limiting the ability to tailor them to students’ needs. As a  result, in developing countries, remedial programs are scarce due to institutional and budgetary constraints, unclear quality standards, and inadequate assessment.

Digital Personalized Learning (DPL) emerges as a promising and cost-effective alternative for math remediation. DPL leverages Artificial Intelligence (AI) and machine learning to provide students with adaptive instruction tailored to their competency levels, known as "Teaching at the Right Level" (TARL).  The basic principle of TARL is to adapt instruction to match students' needs based on their prior knowledge. This adaptation enhances knowledge retention and motivation, while providing a strong foundation for future learning. Adaptive Learning is a promising mechanism to improve student skills and their perceptions about those skills, known as perceived self-efficacy, which is often associated with academic performance, especially in mathematics. DPL also offers pedagogical strategies and regular data for assessment, accessible through various devices with internet access.

A systematic review indicates that DPL-based remediation outperforms traditional methods involving tutors and non-adaptive computer-assisted remediation across education levels. Given the significant expenses associated with tutors and setting up remedial classes, these findings are particularly relevant for college remediation settings. However, most studies on DPL effectiveness in post-secondary education are from developed countries, leaving a gap in the literature.

New experimental results for Ecuador, involving over 11,400 students and 71 technical universities, provide unique evidence supporting DPL’s viability in developing countries. The study conducted a randomized controlled trial during the COVID-19 pandemic, assessing the impact of a DPL software called ALEKS offered to first-year students entering technical and technological higher education programs. Inadequate academic readiness is one of the main causes of college dropout in Ecuador, where 33 percent of new college students drop out within a year.

The results show that the possibility to use the software for about 4 months led to a large and marginally significant decline in the probability of repeating a course, as well as a very large positive impact on standardized test scores in math. On average, students in the treatment group scored 0.28 standard deviations higher than those in the control group, with statistical significance at a 1 percent level. Furthermore, the results also indicated that benefits are proportional to the time students spent using the platform (Figure 2).

Figure 2: Marginal effects of platform time use on mathematics

Overall, the study demonstrates that in just four months, students were able to catch up by the equivalent of about two years of learning. When disaggregating the impacts, the findings show that the effects on repetition are particularly significant for male students, possibly because of higher male enrollment in science, technology, engineering, and mathematics disciplines. The program’s cost was approximately $18 per student, factoring in the number of licenses purchased, the number of teachers trained, and the expenses associated with monitoring the program during its implementation period.

In conclusion, the use of Digital Personalized Learning emerges as a promising and cost-effective solution to address the pervasive issue of skills gaps in mathematics, particularly in the context of developing countries.  In a global education landscape grappling with academic preparedness challenges and the high costs associated with traditional remedial programs, DPL offers an innovative and more accessible pathway forward. By harnessing AI and adaptive instruction, DPL tailors learning experiences to individual students, ensuring that they receive precisely what they need at the right level. The results of the experimental study in Ecuador illustrates the potential of DPL, where a relatively short intervention period resulted in remarkable academic improvements and call for additional evidence from different contexts and circumstances.