A light-touch method to improve accurate reporting of IDP’s food consumption

To design effective and durable relief programs for refugees and internally displaced people (IDPs), it is essential to understand the nature and context of the challenges the people living in these situations face. That’s why we have recently started to measure consumption and estimate rates of poverty among displaced populations. Through understanding the most acute challenges that vulnerable populations face, relief can be targeted to where support is needed most.

Working in such demanding circumstances poses challenges. In a recent survey of IDPs in Somalia, 45 percent of IDP households reported food consumption below the survival rate, even in the short term. About 80 percent reported food consumption below levels that allow normal physical activities without negative health impacts.

The rationale for studying IDPs in Somalia is that they are living in extraordinarily harsh conditions. We worried, however, that the peculiarly low rates we observed may reflect misreporting and measurement errors. For example, respondents may wish to hide what they have fearing that it will be stolen. Or respondents – especially the ones depending on food aid – may expect that reporting lower consumption could lead to more aid. If this is true, it drastically undermines the quality of the data and our ability to design effective programs based on the data.

Given the sensitivity of the situation, we wanted to find a light-touch method for improving the accuracy of such surveys. To understand whether misreporting contributes to the very low levels of consumption, we designed a survey experiment as part of an IDP survey for selected South Sudanese IDP camps. While we measured consumption with traditional methods for half of the surveyed population, the other half was given a bundle of nudges that stressed on the need for accuracy in the reporting:

1. We instructed the interviewer to thank the respondent for the participation while stressing how important accurate information is to understand the situation in South Sudan.
2. We presented the respondent with a short vignette in which one person lies to another and ask the respondent to indicate whether the person did the right thing. By encouraging the respondent to think about the value of honesty, we hoped to improve accurate reporting, as shown in other studies.
3. Finally, we introduced consistency checks by asking about consumption for broad food categories before item-specific consumption (e.g., have you eaten any grains in the last week) and double checking when people reported zero consumption. This form of investigative probing puts a higher salience on reports indicating zero consumption.

We expected this procedure to result in higher consumption reporting for households that otherwise report very low consumption. We investigated the impact of these nudges on four measures of food consumption: the number of consumed items, the consumed quantity in kilograms, the monetary value of consumption and consumed calories. We isolated the impact of the nudges relative to the survey population receiving the standard questionnaire. But because of our hypothesis indicating that the poorest households will increase reporting most when subject to the nudges, we estimated the impact depending on how poor households were.



Confirming our expectations, we found that especially households with lower consumption reported significantly higher consumption when being subject to the nudges. Our light-touch change in the questionnaire shifts 7 percent of respondents above the threshold of recommended daily intake, providing further indication that misreporting existed. Interestingly, we also find that the new interventions do not show significant effects among non-IDP populations, known to be less dependent on food aid. Furthermore, the treatment is more effective for IDPs previously relying on aid, thus, having stronger incentives to misreport.

Due to the low costs, this light-touch intervention can be easily implemented in surveys to improve accurate responses when incentives to misreport exist. Yet we believe that questionnaire design can be substantially improved if tailored and tested for a specific context before the implementation of the survey. Additional research could also help to disentangle the impact of each individual approach within the bundle, and ideally propose additional interventions.