Online pundits, hurried journalists and policymakers love precision. They crave numbers. Preferably exact numbers; ranges suggest uncertainty and make them anxious. As a result, they will love the World Poverty Clock (WPC), a new website that claims to track progress towards ending global poverty in real time (see also this blog and Financial Times article). The website tells you that 632,470,507 people are currently living in extreme poverty - or were, on December 6 at 10:00am… Even more amazingly, the site claims to forecast poverty at any point in the future until 2030, the deadline for the UN’s Sustainable Development Goals. By scrolling along the elegant timeline on the bottom of the WPC screen you will learn, for example, that in 2028, 459,309,506 people will be living in extreme poverty!
If you pause for a moment to consider the myriad sources of uncertainty about what may happen to our world in the next ten years, this must strike you as a remarkably precise forecast. And it is presented down to the decimal point, with no indication of possible error, or of the margin of confidence viewers should place on it. Because, as its ‘methodology’ tab reluctantly admits, the WPC is based primarily on poverty data from the World Bank’s PovcalNet, with which we are involved, we felt it was probably incumbent on us to clarify a few points.
First, although the team behind the WPC does include one World Bank staff member, the WPC estimates were not produced or vetted by the World Bank teams responsible for monitoring global poverty. In fact, we were never consulted about the data or methods. That said, we are always happy when the poverty data we curate and assemble is used by others to focus public attention on the plight of the extreme poor – those living on $1.90 per day or less – and on the challenge the world community faces in meeting SDG 1.1. Of course, long before the WPC came along, a number of studies had already highlighted that reaching the 3% goal by 2030 is far from guaranteed (e.g. Ravallion, Edward & Sumner).
Clever uses of global poverty data for advocacy and public debate are a great thing. That is exactly why the founders of PovcalNet made it a public tool, to begin with. But, to be useful, advocacy must be honest and, in forecasting, honesty requires at least a discussion of the sources of uncertainty associated with one’s predictions. As Tony Atkinson reminded us in his report of the Commission on Global Poverty, even our global estimates of past poverty are subject to numerous sources of error (World Bank, 2017, p. 49). These include the usual sampling errors in the underlying household surveys, but also inaccuracies in population statistics, errors in the ‘lining up’ procedure, errors embodied in Purchasing Power Parity exchange rates and in national consumer price indices, and so on. In other words, even in the latest year for which we have actual data, our estimate of global poverty is imprecise. For this reason, we tend to report the number of poor with one or two decimals on the million (e.g. World Bank, 2016, p. 4).
Even that is probably too much precision, and we plan to be even more explicit about uncertainty in the future. We have committed, given the necessary resources, to follow the recommendation of the Atkinson Commission to estimate the statistical confidence band around our poverty numbers, arising from the sum of all those different sources of error, and to report it alongside our point estimates. Because most of that error is unlikely to come from sampling, which statistics can handle more easily, that is easier said than done. It requires a level of in-house statistical capacity that we still need to invest in. (For starters, even an accurate estimate of sampling error requires information on complex survey design, which countries seldom make available to us.) But to be perfectly clear, the direction of travel is towards more - not less - transparency about the inevitable uncertainty surrounding our poverty estimates – even about the past!
Now, when one goes beyond the past, and seeks to project into the future, estimates naturally become even less precise. This is because, in addition to measurement error (both sampling and non-sampling), forecasts require modeling (however simple), and thus involve modeling error. Models may be incorrectly specified in two broad sorts of ways: first, they might include the right variables but get their functional forms and coefficients wrong. Second, and more worryingly, models may altogether fail to allow for the possibility of crises, epidemics and other unforeseen events that may turn out to matter a great deal. Try, for example, to look for macroeconomic projections for 2008/2009 in, say, the US, that were published one or two years before the onset of the last global financial crisis, in December 2007… Or for growth and poverty forecasts for Sierra Leone published before the onset of the 2014 Ebola epidemic in West Africa...
Poverty projections, specifically, require forecasts for at least three key ingredients: National population, mean income and its distribution. Demographic trends can be predicted relatively precisely, but growth forecasts are often (or should we say, almost always) wrong. See, for example, Figure 2 in this blog comparing IMF global GDP growth forecasts against actual growth. Furthermore, it is typically assumed that mean incomes grow at whatever rate is forecast for national GDP, although it is well known that growth rates in GDP and surveys can – and often do – diverge.
As for changes in distribution, poverty projections typically assume that national inequality remains unchanged. As there exist no standard projections of inequality (because it is extremely difficult to model and forecast), such a simplification is understandable. But it is unlikely to be correct. At the very least, one ought to check for the sensitivity of the poverty forecast to alternative paths for national inequality, as Lakner, Negre and Prydz do in a tool that simulates poverty headcounts after users specify a growth rate and its degree of pro-poorness. A key difference between this online tool and the WPC is that the World Bank site makes it explicit that one is looking at different scenarios, given certain distribution and growth assumptions made by the user. Although there are (still) no confidence intervals around each individual scenario, the range of outcomes arising from differences across scenarios is itself a powerful indication of the uncertainty inherent in poverty projections.
Fighting poverty is important work. Those planning future action to combat it deserve to be told what we can and cannot confidently predict. The fact that there are some precision addicts out there is no excuse for feeding their craving. We would like to be able to welcome the World Poverty Clock as a useful tool for advocacy and debate. But that will require a great deal more circumspection about the uncertainty associated with its forecasts. And, while its makers are at it, they can probably be a little more open about their data sources, assumptions, and methods too.
Join the Conversation