Practical tips for implementing remote surveys in the time of the Great Lockdown

From David et al 2 part blog post (part 1 and part 2) last week, you now know how to construct your sample, what kind of response rate you can expect, how to modify your survey instrument, and how to monitor data quality. But how do you actually get the survey to the field — in a country under lockdown? 

With the Great Lockdown being extended in many countries, and large gains from surveying vulnerable populations to calibrate policy responses, we wanted to provide tips to successfully running a remote survey when enumerators are confined to their homes. Even though you may have hired a firm to run your survey, these are important aspects of field (home) work that need to be taken into account to ensure quality of data. We based this on our recent and ongoing experience of transitioning an ongoing in-person survey to a phone survey with Innovations for Poverty Action in Rwanda — where a lockdown was announced and made effective on March 21st, leaving no time for advance preparations. (The following recommendations work along DIME Analytics checklist for primary data collection.)

logistics, logistics, logistics

In countries under strict lockdowns, simply supplying enumerators with the materials they need to successfully conduct a phone survey will be a challenge—a few of them below: 

• Ideally: enumerators will be supplied with tablets and wifi hotspots. 
• In practice: relying on a selection of enumerators with smartphones, using mobile data networks likely ends up being next best.
• Using the same smartphone for calling and to enter responses can pose a challenge. There are a few technological solutions out there (such as the SurveyCTO CATI starter kit) to reduce the challenge. As our survey was already ongoing, enumerators had access to tablets which they used for data entry while using their phones to call the enumerators.  
• While this is true of any survey, it becomes particularly important to ensure timely transfer of airtime and reimbursements to the enumerators (receipts transmitted by photo) to enumerators is essential. Mobile money is a great avenue for this.
• As we were simply adapting an existing survey instrument, we did not do a full pilot. If needed, you will have to have access to contact information for out-of-sample respondents. If that’s not available, random digit dialing is an option. 

“e-training”

While we expected that recreating a classroom training experience virtually would be a challenge, we underestimated just how difficult it would be. Again, it is important to get a reality check on what media your enumerators will have access to, and limit the complexity of your survey accordingly. Here are a few hard-learned lessons:

• Share training materials with the survey team in advance, and make sure they are in a format easily opened on a phone (for example, pdf will work better than powerpoint). Keep training materials as simple as possible, and compress or send in many batches, to avoid heavy attachments. 
• Make plan A, B, and C: video conferencing is ideal, to maintain the personal connection, easily show examples, and take advantage of interactive features. In our case, video connect was impossible due to limited bandwidth. We planned an audio only training and re-framed materials accordingly. Yet the enumerators still had a lot of trouble with the connection; calls were dropped, and audio quality cut in and out. Try different platforms or a combination of platforms to see what works best. 
• Along these lines, we strongly advise conducting the training in small groups: although repetitive for the trainer(s), this reduces connectivity challenges (or at least makes it easier to work around people dropping in and out) and allows for more interaction.
• The ideal training schedule will look quite different than an in-person training. Training straight from 9-5 is clearly impractical. We found training in 2-3 hour chunks, spread over the course of a few days, to be most effective. The ideal training times may be different than you’d expect: evening hours might be the best for uninterrupted stretches (after children have gone to bed and all meals are prepared)--it is important to check with the enumerators and trainers and potentially accommodate different schedules for different groups.
• Establish clear conference call protocols and distribute to the enumerators in advance of the training: see this guide for an example.
• Consider pre-recording training sessions, in anticipation of connectivity problems. While transferring large files is impractical, uploading to an online platform can be a better option. This is also useful as backup, in case an enumerator does get dropped from a call, or has trouble hearing. 
• Mock interviews are a great tool! Have enumerators submit that data to the server, so the team can review and provide custom feedback. We suggest using pre-recorded interviews, and playing them for all enumerators, who enter data based on what they hear and submit it to the server. It’s then easy to review and “grade” the responses. Conducting quizzes via short survey forms is another good way to assess comprehension.
• While this is always true, working with experienced enumerators will make training in these conditions much more effective—if this is a follow-up survey, re-hiring enumerator from the previous round is ideal.

talk to me!

Establishing good feedback loops and communication channels will be harder with enumerators working remotely. We found the following to be particularly helpful: 

• Have a clear first point of contact at every level of the survey hierarchy. Problems will inevitably arise, and everyone needs to know who to talk to, and how to reach them quickly.
• Set up a clear system for tracking progress. Be mindful of what technology enumerators have access to: an Excel spreadsheet may only be possible to open if they own a computer, and access to a printer is unlikely. We settled on a simple form enumerators fill on paper by hand as they go, and then take a picture of and send to their supervisor at the end of each day using WhatsApp. 
• Debrief meetings are even more important when teams are not interacting directly. Lots of useful conversation happens traveling to sample locations, over lunch, while waiting for other team members to finish, etc. It’s important to intentionally create space for team members to ask questions, share stories of unusual or unexpected experiences, and ask each other for advice. 
• Run data quality checks frequently, and communicate results to enumerators promptly. Use a combination of data quality checks and do not rely solely on audio audits as there are limitations imposed on recording phone calls by Google and Android (if using the same device for data entry and calling), audio quality can be poor (dependent on microphone quality of the smartphone) and, unless enumerators use speaker phone (which drains battery and may raise privacy concerns), only one side of the conversation is captured…

safe data

The data may have to be stored on private phones until transmitted to the server … this poses a serious threat to data safety—we recommend the following:

• Password protect the devices used for data collection — if enumerators are using their personal phones, ask them to verify that they will be using a password to protect access
• Ask enumerators to tear and throw away tracking sheets and delete any personal information of respondents on their phones at the end of the survey
• Encrypt the survey forms.
• Have the enumerators sign a non-disclosure agreement (digitally or on paper and send a picture) is recommended. This is to ensure the respondent contact numbers do not get used for any other purpose.

Good luck everyone, and stay safe!

We acknowledge the central role IPA Rwanda played in delivering these solutions, in particular Phillip Okull, Doug Kirke-Smith, and the enumerators' patience and persistence (pictured) in getting this to work.