Supporting learning continuity: Insights on the use of mobile phones for formative assessment

When the COVID-19 pandemic started in March 2020, most countries closed schools for in-person instruction to prevent the spread of the virus. Even today, more than two years since the pandemic started, schools in some countries are still partially closed or have faced intermittent interruptions to in-person instruction. World Bank simulations and learning assessment results in countries around the world indicate that students are returning to schools with learning losses. If governments do not take remedial actions to support students, teachers, and schools, these losses may have permanent impacts on individuals and societies in the future.

The World Bank’s Learning Assessment Platform (LeAP) team has been working on evaluating the feasibility of phone-based formative learning assessment activities as a response to the learning crisis exacerbated by the COVID-19 pandemic. The objective of this activity, financed by the Continuous and Accelerated Learning (CAL) grant from Global Partnership for Education, is to foster learning continuity, particularly in contexts with limited connectivity and access to smart digital devices.

In general, formative learning assessment activities—also known as “formative assessment” or “assessment for learning”—allow teachers to determine what students know, understand, and can do, often as part of regular classroom instruction. This type of low-stakes assessment is essential for fostering student learning as teachers can use results from these assessments to adjust classroom instruction, provide constructive feedback to students, guide them in their learning, and offer additional opportunities to strengthen their knowledge and skills. Contrary to the annual frequency of most large-scale assessment and high-stakes examination programs, formative assessment is conducted daily in the classroom as a core element of the teaching and learning process.

While not a common modality for administering formative assessment activities, the relatively wide coverage of basic mobile phones, even in low-resource contexts, makes them an attractive option for complementing in-person formative assessment activities, leveraging their functionality in terms of live phone calls, short message services (SMS), and interactive voice responses (IVR).

As part of the CAL grant activity on phone-based formative assessment, the LeAP team has developed a set of resources to guide those interested in implementing formative assessments using basic mobile phones to support learning continuity, including:

• Guidance note on key elements necessary for implementation and scale-up of phone-based formative assessment approaches in low-resource contexts;
• Landscape review database of existing phone-based assessment interventions and their key features;
• Note on psychometric considerations for implementing phone-based learning assessments and interpreting their results;
• Template for implementation feasibility to assess prerequisites and enabling conditions for each of the phone-based formative assessment solutions; and
• Knowledge pack in collaboration with the World Bank’s EdTech team to provide key insights on phone-based assessments to non-technical audiences.

The insights from the above resources have informed the pilots of phone-based formative assessment in Ghana, Nepal, and Pakistan, conducted as part of the CAL grant activities. These pilots have resulted in two additional knowledge products:

• Synthesis report to describe the phone-based assessment implementation process and lessons learned in three countries: Ghana, Nepal, and Pakistan; and
• Package of remote formative assessment tools, developed for the Ghana, Nepal, and Pakistan pilots and made publicly available for stakeholders interested in reviewing and/or adapting assessment instruments for remote administration over the phone.

There are several key lessons emerging from this work on the use of basic mobile phones to carry out formative assessment activities:

• Instructional coherence: To support learning, the content of the formative assessment conducted over the phone must be well aligned with the learning standards described in the curriculum. This coherence will make assessment results more actionable for teachers and parents and will support students’ progress along their learning trajectories.
• Local adaptation: Critical contextual factors that should be considered when thinking about the implementation feasibility of phone-based formative assessment include demographics (for instance, local languages spoken at home and literacy levels among adults in the household), access to mobile technologies, and the capacity of local service providers. Local technological and logistical considerations include access to electricity at home and available mobile infrastructure (such as mobile aggregators with bulk SMS and IVR capacities), the timeline of the school calendar, availability of financial and human resources, as well as the capacity of local implementing organizations with relevant experience in phone-based data collection. Likewise, cultural and behavioral aspects can affect the effective implementation of phone-based formative assessment, including the parents’ views on phone-based learning activities and any differences in access to the phone between girls and boys.
• Multilingual support: Multilingual settings can increase the complexity of administering phone-based assessments, but some phone-based solutions, such as IVR and live phone calls, may offer promise to reach students in the language they speak and understand. Take-up of the assessment may also increase when the initial engagement with caregivers and children occurs in the language they speak at home, even if that is different from the language of instruction and assessment.
• Appropriate use: A phone-based assessment modality is appropriate for low-stakes formative assessment activities, where the purpose is to support individual learners. However, these solutions would generally not be appropriate for high-stakes decisions or system-level monitoring activities, as it would be very difficult to ensure test security and uniform administration needed for any high-stakes exam or to gather a representative sample of students of a certain grade or age necessary for a large-scale assessment.

These lessons, and the global public goods developed under this work by the Bank’s LeaP team, can be valuable for other countries and organizations interested in using phone-based learning assessments to support accelerated learning recovery from COVID-19 and promote education systems’ resiliency to future shocks.