One of the first Millennium Development Goals is to reduce hunger by half between 1990 and 2015. To date, the global hunger count has fallen slightly, from 1 billion in 1990–1992 to 870 million in 2010–2012 (Food and Agriculture Organization 2013). As a proportion of the world’s population, this is just a one-third fall in the hunger rate, from 19% to 13%. In contrast, the other highly visible Millennium Development Goal – reducing extreme poverty by half – was achieved by 2010.
The failure of the hunger rate to fall as fast as poverty undoubtedly has many causes. Increased volatility of world food prices and the damaging responses by some countries hardly help (Rocha et al. 2012). But researchers also are starting to study measurement issues – the methodology for measuring the hungry differs from that for measuring the poor.
The Food and Agriculture Organization measures hunger by combining aggregate Food Balance Sheets (FBS) for every country with selected household survey estimates of the inter-household variance in calorie availability. Using an assumed distribution, with the mean calorie availability from the FBS and the variance from the surveys, the population falling below calorie requirements is estimated. This ‘top-down’ method has been criticised by researchers (e.g. De Haen et al. 2011), and contrasts with the ‘bottom-up’ approach of the World Bank, where global poverty counts are constructed directly from household surveys.
Recent research advocates calculating hunger numbers directly from household consumption expenditure surveys, or HCESs (Fiedler 2012). We call this the ‘HCES-direct’ approach, and contrast it with the ‘FBS-CV’ approach of the Food and Agriculture Organization, where surveys just contribute the coefficient of variation of calorie availability, and the remaining information is from the Food Balance Sheets.
The confounding effect of variation in survey design
While there may be good reasons to switch from the FBS-CV approach to the HCES-direct method, both are subject to measurement concerns due to variation in household survey design over time and space. Unlike national accounts, which benefit from internationally coordinated development and updating, there is no coordinated approach to HCES development. A lack of harmonisation is not inherent in micro survey data – the standardised approach of the Demographic and Health Surveys is an example of successful harmonisation. Instead, the diverse constituencies served by HCES and lock-in of some initial idiosyncratic variation matters here.
To provide evidence on these measurement concerns, we randomly assigned seven different HCES designs to 3,524 households in 168 communities in Tanzania (De Weerdt et al. 2014). The designs varied in four key dimensions:
•Data capture (diary versus recall);
•Level of respondent (individual versus household);
•Reference period for which consumption is reported (from seven days to ‘usual month’); and
•The degree of commodity detail (from 17 to 58 food groups).
The seven modules broadly represent the current variation in HCES modules across countries and time. No method relies on individual recall of meals eaten (a food intake survey), as that method is not typically used with these national household surveys.
The experiment covers urban and rural settings, and reflects the range of Sub-Saharan conditions where, according to the Food and Agriculture Organization, the proportion of hungry people is highest and hunger prevalence is increasing. Within communities, households were randomly assigned to one of the seven survey variants. Fieldwork covered 12 months, ensuring that any differences in derived hunger numbers between HCES methods are solely attributable to the survey design, as opposed to persistent differences across communities or seasons.
The food quantities reported in the surveys were transformed into food energy availability using food composition tables. The per capita daily (kilo)calories available are further adjusted for meals eaten out of the home and meals shared with non-household members.
Variation in estimated hunger prevalence
Table 1 shows great variability from different survey methods, with reported calorie availability ranging from 1,793 to 2,677 per person per day. This wide range of estimates – varying by a factor of 1.5 from highest to lowest – is twice the range for monetary measures of consumption from the same experiment (Beegle et al. 2012), although closer to the factor by which food consumption varies (1.3). Estimates of total consumption are less sensitive to design variation because non-food consumption measurement was standardised in the experiment, reflecting greater uniformity in design of questions on non-food expenditure than on food consumption in household surveys.
Table 1. Design of each household consumption expenditure survey and calorie availability
* The 17 foods account for 77% of the food budget, so calorie availability is scaled up by 1/0.77.
We characterise a household as hungry if total dietary energy available is lower than the energy requirement of residents, accounting for their age, gender, and the likelihood a women is breastfeeding. This is the direct method of calculating hunger from household surveys that some researchers are advocating. However, as Figure 1 shows, this HCES-direct measure is not robust to variations in survey design. The estimates of hunger prevalence (the percentage of individuals living in hungry households) range from 19% to 68% (and the 95% confidence intervals are around nine ercentage points, so most of the differences are highly significant).
Figure 1. Hunger prevalence, HCES-direct method (% of people in households with energy availability below requirements)
The potential for noise in hunger counts from a direct measure, stemming from alternative survey designs, is clear. But this fragility also affects the results of the FBS-CV method. We show two sets of results for this method in Figure 2. First we assume that different survey designs do not affect the estimate of mean calorie availability from the FBS, and that somehow the FBS could produce the same ‘correct’ mean of 2,677 calories per capita per day given by the individual diary (which is presumed to be closest to the truth). This assumption is unlikely to be realistic, since some of the inputs to the FBS come from surveys, and these are likely to vary across countries in similar ways to the variation in the HCES. Yet even with this assumption, the prevalence of hunger varies between 17% and 28%.
The other results in Figure 2 set the mean calorie availability to the module-specific results from Table 1, with the Food and Agriculture Organization assumptions then applied to the coefficient of variation of calorie availability from the alternative survey designs. In other words, we assume that the estimates of national average calorie availability from the FBS vary between countries at the rate in which our experiment shows variation in the mean. This is also unlikely, since some inputs into the FBS come from relatively harmonised sources such as international trade statistics, while others come from less harmonised sources such as agricultural and household surveys. In this case, the estimates of the hunger rate would be just as variable as in Figure 1, ranging from 19% to 72%. In reality, the FBS-CV method is likely to yield results for hunger prevalence that are somewhere between the two extremes shown in Figure 2, which still implies great sensitivity to measurement methods.
Figure 2. Hunger prevalence, FBS-CV method (% of people in households with energy availability below requirements)
Our survey experiment varies how food consumption is measured and yields vastly different hunger estimates, ranging from 19% to 68% of the population being hungry. That is a difference of more than 23 million people in Tanzania. Across all of Sub-Saharan Africa, these differences would amount to hundreds of millions of people potentially misclassified as either hungry or not hungry depending on what survey method is used.
While we believe that the calculation of hunger numbers directly from household surveys holds great promise, this approach may only be feasible once much more effort is put into harmonising household survey design around the globe. Until that time, comparisons of hunger across countries and over time should be considered as fraught with measurement error.
The above post first appeared in VoxEu.
Join the Conversation