Designing programs and policies to eliminate extreme poverty and boost the income of the bottom 40% of the world’s population requires reliable data. The data is gathered through using household surveys that typically collect information on consumption and expenditure on a comprehensive set of 300 to 600 items in the household. A list of items includes anything that a household potentially consumes, for example, typical items like maize, milk, and soap, as well as rare items like mustard.
Consequently, such household interviews are time-consuming, taking anywhere between 90 minutes and several hours to complete. Because of security considerations, carrying out these surveys is not always feasible, especially in insecure and fragile contexts. To mitigate security risks, the time to interview a household must be limited with field operations designed around small and agile teams.
We encountered such a situation in Somalia, where we were restricted to 60 minutes to conduct a household interview. Faced with this constraint, we had to develop and implement a new approach.
The most straightforward approach is the reduced consumption methodology. As the name suggests, this reduces the number of consumption items by either asking for aggregates or skipping the less frequently consumed items. Unsurprisingly, this method is consistently biased to underestimate consumption and, therefore, overestimate poverty.
Forced to come up with something better, we developed the Rapid Consumption Survey methodology. The Rapid Consumption Survey methodology combines an innovative questionnaire design with statistical imputation techniques. Instead of assigning all consumption items to all households, important items are assigned to a core module, while the remaining items are split into four or more optional modules.
Each household reports on the core module, as well as on one of the optional modules. This reduces the interview time considerably, down to 45 to 60 minutes per household. We pay for this gain by accepting the missing information from the optional modules that were not administered to the household. However, we offset that by administering those optional modules to other households. Thus, we are able to estimate the missing information for one household based on the information collected from other households.
Once we came up with the methodology, we had to test it before implementing the approach in the field. Therefore, we used a survey from 2013 that was restricted to Somaliland and simulated the Rapid Consumption Survey methodology. We compared the new approach with the full consumption as reference. The new approach estimated poverty virtually unbiased (well below 1 percent difference to the reference).
We were also curious to find out if we would have achieved the same result with the reduced consumption approach? The answer was a clear “no,” but it needed some explanation. Assuming that we removed the items that had the least consumption, the reduced consumption approach actually didn’t perform too badly, underestimating consumption by only about 4%.
In reality though, we do not know which items are consumed the least. In fact, we found that it was a very bad idea to infer those items even from “similar” cities. The 33 most consumed food items in Hargeisa cover 91% of food consumption. However, the same items only cover 54% of food consumption in Mogadishu. Thus, we would have grossly underestimated consumption if we had used the reduced consumption methodology in Mogadishu based on the items from Hargeisa.
Based on the simulation results and the pilot in Mogadishu, we embarked on implementing the Rapid Consumption Survey methodology in several countries, including Somalia, South Sudan, and Kenya. In Somalia, the survey was implemented in the spring of 2016 delivered the first ever monetary poverty estimates indicating that 51% of the covered population are poor (using the international US$1.90 PPP poverty line). The estimates and related analyses were featured prominently in the National Development Plan influencing policies and programs.
In South Sudan, the new methodology allowed us to track poverty over time. In combination with our real-time market price dashboard, this helped to understand the impact of rising inflation to help us inform policy responses. We also captured video messages from affected people which helped us provide a more subjective view from people’s personal experiences in South Sudan.
While the fragile context forced us to innovate, we think the new approach has wider use. This methodology could be implemented even under normal settings, where survey respondents often get bored with long interviews, a factor that increases measurement error. We are thinking about applying the same logic, called a split-questionnaire design, to modules other than consumption.
As we gain insights, we will share what we learn. Stay tuned.
Join the Conversation