Remote data-collection during COVID 19: Thing of the past or the way of the future?

The COVID-19 pandemic pushed the process of collecting learning data from in-person to remote in just a few weeks, mobilizing researchers and practitioners alike to learn new systems, adapt instruments, and engage in a great deal of trial and error. Today, nearly two years after the first shutdowns, the education sector is cautiously moving back to in-person activities with new expertise and new questions, including: “What will we continue to do remotely? What can’t we do remotely? What does this mean for our future interventions?”  The answers will vary depending on organization and context.

In this blog, we outline the successes, failures, and opportunities from remotely enrolling participants and surveying for the SMS-based program in northern Ghana, part of a randomized control trial funded by the Strategic Impact Evaluation Fund (SIEF).

We partnered with Innovations for Poverty Action (IPA) and Movva Technologies to develop and test the impact of short behavioral “nudges” to parents in the form of text messages. These nudges were designed based on an existing evidence-based program (Ready4K!) and adapted to the Ghanaian context with the goal of improving parental engagement in educational activities and parental beliefs in the value of education - with a specific focus on gender equity. Although the intervention was designed for remote implementation, the data-collection processes were not, which required quick adjustments to the enrollment and piloting processes in the wake of the COVID-19 pandemic.

Enrolling Participants

We were able to get the phone numbers of potential participants from a previous similar study. A dedicated team of enumerators then proceeded to reach out to households and see if they were interested in participating in the program. Enrolling over the phone is not new but is especially challenging in rural areas like Northern Ghana where over half of the households ended up being unreachable by phone, demanding the need for some in-person tracking. However, some small changes made a big difference in making the phone-enrollment process more successful.

Trust: Trust is an essential part of successfully enrolling participants in an intervention, especially one with a research component that requires consent.  In Ghana, spam phone calls and texts are common, making the transition to phone recruitment difficult as unknown numbers can’t be trusted. To overcome the suspicion from participants, enumerators built trust over the phone by:

• The enumerator gave the number of the Research Associate to participants so they could verify if the calls were authentic and asked the participants to save their numbers in their phone. Having these numbers saved in the participants phone meant avoiding future screening from participants, making it much easier to reach them for follow-up surveys.
• The website of the organization was also shared for persons in doubt.
• These surveyors could speak to the prior projects in a way that let the caregivers know they were authentic.

Connectivity: The largest barrier to remote data collection was network connectivity. Even after successful recruitment, it was difficult to reliably reach participants for baseline data collection due to network connectivity. The limitations of phone-based surveys and assessments became quickly apparent as broken phone connections resulted in half-surveys and frustrated participants.  One way of overcoming this challenge was to call on market days when participants were going to more central locations with better connectivity. In some cases, enumerators had to rely on traditional in-person tracking to make sure all data was collected.

Enumerator Training: Enumerators receive extensive training to ensure data quality, but COVID meant adapting the training to an online platform. Poor connectivity was a challenge as some participants cut in and out. Additionally, trainers were unable to physically monitor interviewer performance. With such a large sample, the risk of invalid data collection resulted in the team making the difficult decision to move some of the assessments, such as the child-level assessments, to a later date to make sure they were in person.

Conclusion

The COVID pandemic, and resulting lack of access to participants, will mean needing to continually improve how everyone in the education sector collect data remotely. The lessons learned during the move to remote data collection offer opportunities to invest in and improve how we collect data remotely, potentially opening opportunities to reach more people  in difficult-to-reach parts of the world or during emergencies.

While we still have a long way to go to find remote and resource-efficient methods that can adequately replace in-person interactions. Our experience in northern Ghana shows that even small changes can make a big difference in reaching participants and shows us where future investments will make the most impact.  For example, while phone surveys with youth have been shown to be quite successful, making this something available for younger children is a challenge that we hope to be part of addressing in the future. The conversations around how, when, and where to conduct remote data collection will undoubtedly be explored for years to come, as there are advantages to phone surveys, particularly because of much lower costs than in-person assessments.

The pandemic has imposed a “sink-or-swim” reality on the entire data collection sector, forcing it to determine very quickly what can be done remotely and what, given our current access to technological resources, cannot. Moving forward, certain elements of enrollment may be continued remotely, allowing for saving scarce resource and further reach of participants. Improved online modes of interaction that build trust and rapport will be needed to make the job of implementing a research-practice intervention something that can be done entirely remotely.