In 2017, a severe and prolonged drought had hit countries in Africa and the Middle East, bringing crop shortage, livestock death, water scarcity and disease. Food shortages escalated into near-famine conditions in countries with low resilience against shocks, such as Nigeria, Somalia, South Sudan and Yemen. In such a context, rapid quantitative data is required to respond to urgent developmental needs of the affected populations. Therefore, we designed and implemented the Rapid Emergency Response Survey (RERS).
Through the RERS, we were able to find that more than 7 in 10 households of the survey population are highly food insecure in South Sudan. In Nigeria and Yemen, every second household is highly food insecure, while 3 in 10 households in the Somali population face high food insecurity. Food insecurity is worse in the countries that faced conflict during the crisis. High incidence of conflict was reported in Nigeria, South Sudan and Yemen, while for the Somali population, the crisis was primarily due to dry agricultural seasons and a lack of resilience.
Apart from food insecurity, the populations faced a range of developmental challenges. Livelihoods were affected in all four countries, with large portions of the populations (ranging from 31 percent in Nigeria to 84 percent in Yemen) facing a decrease in income and changing the main source of livelihood. Poor health, insufficient access to water and low preparedness for the drought were also common to the countries. Other issues like school attendance and livestock loss were more context specific.
Results Dashboard
The results dashboard showcases selected trends for the four countries. In Nigeria, the survey represents the North-East, North-Central and South-South zones. Of these, the North-East zone has states classified under Emergency by the IPC. For the Somali population and South Sudan, only areas declared to be in Emergency or worse are surveyed. In Yemen, the survey covers all regions, stratified into Emergency and non-Emergency. Non-Emergency regions are sampled because they had pockets of highly food-insecure households.
In the context of crises, traditional face-to-face survey methods are not only too costly but also often unsuitable. They take a long time to prepare making it difficult to deliver timely data in a volatile situation. Data collection itself can be risky in crisis locations, for instance if the crisis involves conflict or a health epidemic. The design of the RERS responds to these constraints.
How did we do it?
The RERS is a phone survey, allowing rapid access to crisis-affected populations without exposing field teams to unnecessary risks. Interviews are 20 minutes long to avoid response fatigue and minimize the risk of burdening already stressed respondents with long interviews. Local call-centers are inexpensive to operate, leading to a low overall cost. Enumerators record and upload interviews instantaneously onto a cloud server, allowing for parallel analysis of the data and complete result dashboards within 8 weeks.
A limitation of this approach is that the population that does not use mobile phones, for instance because of poverty or lack of mobile phone coverage, is not represented. Even with this caveat, the RERS allows for an immediate, quantitative assessment of challenges from the crisis. If the survey population is less poor or better-connected, the results of the survey can be interpreted as a conservative estimate of how the entire population is affected.
Sampling strategies are flexible to local constraints. Existing surveys that represent the target population and have phone data, are a very suitable sampling frame. These were available and used in Nigeria. In the absence of existing surveys and reliable phone lists, different strategies can be employed, as we did in the other three countries. In Somalia, a bulk-SMS was sent to mobile customers and a random sample of interested and consenting respondents was selected. In South Sudan and Yemen, random digit dialing (RDD) was used to circumvent the need for a phone list while ensuring random selection of respondents.
Lessons from the implementation
The RERS approach shows us that it is possible to collect quantitative data on populations in crisis, and generate results rapidly. In all four countries, the turn-around from survey inception to results took less than 8 weeks. The data collection phase ranged from 10 days (completing 2600 interviews) in Somalia to 25 days (for 600 interviews) in Nigeria. Team size and response rates were key in determining the speed of data collection—in Somalia, 25 enumerators were deployed, five times the size of the Nigeria team. The Somali case further demonstrates that extremely rapid data collection can be done with a reasonable enumerator team size, even in constrained environments. This rapid implementation was completed at roughly $50,000 per country. The cost per interview was less than $35 in all countries except for Nigeria.
Questionnaire design is a crucial step to accurately capture relevant dimensions of the crisis. Given the crisis was unfolding and with limited existing data, we designed the questionnaire to cover a diverse spectrum of developmental themes. These included education, livelihoods, health, remittances and market access. Given the short interview time, the diversity of themes limited in-depth exploration. Once the data was analyzed, it became clear that the survey population faced more challenges in certain sectors more than others, warranting a more in-depth exploration of those themes to better guide developmental interventions. For example, in South Sudan, more than 90 percent of the households were afflicted by illnesses over the past 3 months. While the questionnaire explored details of the most recent illnesses in the household, the depth of the health module did not allow for drawing specific, targeted health interventions. In hindsight, Additional information on household-member specific and less recent illnesses would have been valuable. The collected data also showed that remittances were not severely affected in South Sudan. Fewer questions on remittances could have made space for the more detailed questions on health. The challenge is that it is not possible to make such choices a priori, especially in data-scarce, unfolding crisis situations.
The way forward
The RERS has shown to gather data in a timely and cost-effective manner under crisis situations but the experience also reveals that we need a system that would help modify questions on the fly based on the data trends. That’s where adaptive questionnaire design can help. The main premise is to allow the questionnaire to change over the course of data collection. The questionnaire starts with a broad range of developmental themes, with preliminary questions about the extent to which the population is affected in each theme. After about 500 interviews, data trends will indicate the developmental themes where the population is affected most. Less relevant themes can be eliminated (marked grey below), and more relevant themes can be explored in increasing detail (marked green). Even saving 5 minutes from skipping preliminary questions on irrelevant themes, creates crucial space for more relevant ones in a rapid, 20-minute interview. The questionnaire can be updated in this way after every 500 interviews.
This design fits well with the RERS methodology that has proven to be feasible. Training enumerators on the full questionnaire beforehand can allow quick adaptation of relevant and irrelevant topics. The design will create systematically missing values for detailed questions in interviews conducted at the beginning of data collection and for explorative questions later in the implementation. The random sequence of interviews, however, ensures that the missing data is not biased and, thus, can be analyzed by ignoring missing values. The call center set-up of the RERS allows for meeting sample size requirements for different topics at virtually no extra cost.
Join the Conversation