The COVID-19 pandemic pushed the process of collecting learning data from in-person to remote in just a few weeks, mobilizing researchers and practitioners alike to learn new systems, adapt instruments, and engage in a great deal of trial and error. Today, nearly two years after the first shutdowns,The answers will vary depending on organization and context.
In this blog, we outline the successes, failures, and opportunities from remotely enrolling participants and surveying for the SMS-based program in northern Ghana, part of a randomized control trial funded by the Strategic Impact Evaluation Fund (SIEF).
We partnered with Innovations for Poverty Action (IPA) and Movva Technologies to develop and test the impact of short behavioral “nudges” to parents in the form of text messages. These nudges were designed based on an existing evidence-based program (Ready4K!) and adapted to the Ghanaian context with the goal of improving parental engagement in educational activities and parental beliefs in the value of education - with a specific focus on gender equity. Although the intervention was designed for remote implementation, the data-collection processes were not, which required quick adjustments to the enrollment and piloting processes in the wake of the COVID-19 pandemic.
We were able to get the phone numbers of potential participants from a previous similar study. A dedicated team of enumerators then proceeded to reach out to households and see if they were interested in participating in the program. Enrolling over the phone is not new but is especially challenging in rural areas like Northern Ghana where over half of the households ended up being unreachable by phone, demanding the need for some in-person tracking. However, some small changes made a big difference in making the phone-enrollment process more successful.
Trust:In Ghana, spam phone calls and texts are common, making the transition to phone recruitment difficult as unknown numbers can’t be trusted. To overcome the suspicion from participants, enumerators built trust over the phone by:
- The enumerator gave the number of the Research Associate to participants so they could verify if the calls were authentic and asked the participants to save their numbers in their phone. Having these numbers saved in the participants phone meant avoiding future screening from participants, making it much easier to reach them for follow-up surveys.
- The website of the organization was also shared for persons in doubt.
- These surveyors could speak to the prior projects in a way that let the caregivers know they were authentic.
Connectivity: The largest barrier to remote data collection was network connectivity. Even after successful recruitment, it was difficult to reliably reach participants for baseline data collection due to network connectivity.One way of overcoming this challenge was to call on market days when participants were going to more central locations with better connectivity. In some cases, enumerators had to rely on traditional in-person tracking to make sure all data was collected.
Enumerator Training: Enumerators receive extensive training to ensure data quality, but COVID meant adapting the training to an online platform. Poor connectivity was a challenge as some participants cut in and out. Additionally, trainers were unable to physically monitor interviewer performance. With such a large sample, the risk of invalid data collection resulted in the team making the difficult decision to move some of the assessments, such as the child-level assessments, to a later date to make sure they were in person.
The COVID pandemic, and resulting lack of access to participants, will mean needing to continually improve how everyone in the education sector collect data remotely.in difficult-to-reach parts of the world or during emergencies.
While we still have a long way to go to find remote and resource-efficient methods that can adequately replace in-person interactions. For example, while phone surveys with youth have been shown to be quite successful, making this something available for younger children is a challenge that we hope to be part of addressing in the future. The conversations around how, when, and where to conduct remote data collection will undoubtedly be explored for years to come, as there are advantages to phone surveys, particularly because of much lower costs than in-person assessments.
The pandemic has imposed a “sink-or-swim” reality on the entire data collection sector, forcing it to determine very quickly what can be done remotely and what, given our current access to technological resources, cannot. Moving forward, certain elements of enrollment may be continued remotely, allowing for saving scarce resource and further reach of participants.