Dear governments: Want to help the poor and transform your economy? Hold on, recalculating…

|

This page in:

You may have heard about Tyler Cowen’s interview with Chris Blattman earlier this month. You would be forgiven, however, to have missed important news about cash transfers because, as far as I can tell, no one tweeted about this:

BLATTMAN: We recently went back to some cash transfers that were given almost 10 years ago, following up a randomized control trial in Uganda in the north, and we’re just, in some sense, putting out those results.

What we found is, the initial result after two and four years was like other places seeing big advances in incomes. People get cash. They’re poor. They couldn’t invest in some of their ideas, but they had good ideas, and so they take off.

Now what we’ve seen is, essentially, they’ve converged with the people who didn’t get the cash. The people who didn’t get the cash have caught up because they saved and accumulated slowly and got up to the point where they have the same levels of success.

They converged to a good level. But this means that cash transfers are much more of a temporary acceleration than they are some sort of permanent solution to poverty.”


Now, I am not surprised at the finding. An increasing number of studies show short-term effects of cash transfers dissipating over time, at varying speeds of decay. But, more on that below… What did surprise me is that I had to read the transcript of the interview to find out about this new finding (no working paper yet, it seems, but here is an abstract). No one was tweeting about the massive four-year effects disappearing: remember that women almost doubled their income compared to the control group five years earlier. It’s not news that these effects are gone?

We are all guilty. If the quote had been about the durability of the effects of cash transfers – even at half of the short- and medium-term levels – many of my tweeps would be shouting it from the rooftops. Why? Because, we disseminate evidence that reinforces our view of the world, but choose to ignore or rationalize away stuff that does not. That may help to keep oneself sane these days, but a good public academic it does not make. Most of us think we’re better than that but we are fooling ourselves. Yes, many of you will politely retweet one of my posts about this or that hype about cash transfers, but deep down you know what you think: unconditional cash transfers are great and there is not a thing any researcher can do about it…

Another study, this one with a working paper, you would not have seen make the rounds on Twitter is the three-year follow-up of the original GiveDirectly experiment, following up on the nine-month impacts published a little over a year ago. Here is an excerpt from page 2:
 

“Comparing recipients households to non-recipients in distant villages, we find that recipients of cash transfers have 40% more assets than control households three years post transfer. This amount (USD 422 PPP) is equivalent to 60% of the initial transfer (USD 709 PPP). However, we do not find statistically significant across-village treatment effects on other outcomes. This difference could stem …from potential spillover effects at the village level. Indeed, non-recipient households in treatment villages show differences to pure control households on several dimensions. The point estimates suggest spillover households spend USD 30 PPP less than pure control households, or about 16% based on a pure control mean of USD 188 PPP, and score ˜0.25 SD less on an index of food security than pure control households. Spillover households also score ˜0.18 SD less on an index of psychological wellbeing than pure control households.”


Two months after being posted online, there are no newspaper articles, blog posts, or Twitter threads about the findings [update (3/27/2018): there was this short blog post from GiveDirectly, published on 2/14/2018]. In contrast, this working paper from the same experiment was made available simultaneously with an article in the Economist (I blogged about it here). Again, why isn’t GiveDirectly’s Twitter feed full of threads about this new paper? Say it with me: “Because, it’s dissonant to the song we want to be singing.” But, if we’re honestly going to talk about publication bias and transparency, we need to disseminate the successes and the failures with equal vigor. Otherwise, what are we doing here?

By the way, what’s the message you should be taking from the more recent paper by Haushofer and Shapiro (2018)? It may be worse than the short-term effects dissipating over time…

Here is a table I put together of the relative means of three outcomes at the three-year follow-up in three counterfactual groups (pure control, treatment, and spillover). As you can see in Table 5 of the paper, the treatment group is not doing better than the control group. Worse, the spillover group is not faring well at all – lower levels of expenditures, food security, and psychological wellbeing that are shown to be statistically significant in Table 7. Spillover effects are larger in absolute magnitude than the ITT effects, meaning that the total causal effect is negative in each case:
 

[The study design-minded readers, please refer to the methodological footnote below][1]

Even in the most favorable interpretation of these new findings, however, the fact remains that there is no treatment effect of cash transfers on beneficiary households other than a sizeable increase in non-land assets, which are held mostly in improved roofs and livestock. This new paper and Blattman’s (forthcoming) work mentioned above join a growing list of papers finding short-term impacts of unconditional cash transfers that fade away over time: Hicks et al. (2017), Brudevold et al. (2017), Baird et al. (2018, supplemental online materials). In fact, the final slide in Hicks et al. states: “Cash effects dissipate quickly, similar to Brudevold et al. (2017), but different to Blattman et al. (2014).” If only they were presenting a couple of months later…

Cash transfers do have a lot of beneficial effects – depending on the target group, accompanying interventions, and various design parameters, but that discussion is for my next post…

 
 
[1] The short-term impacts in Haushofer and Shapiro (2016) were calculated using within-village comparisons, which was a big problem for an intervention with possibility of spillovers, on which the authors had to do a lot of work earlier (see section IV.B in that paper) and in the recent paper. They got around this problem by arguing that spillover effects were small and insignificant. Of course, then came the working paper on negative spillovers on psychological wellbeing mentioned above and now, the spillover effects look sustained and large and unfortunately negative on multiple domains three years post transfers.

The authors estimated program impacts by comparing T to S, instead of the standard comparison of T to C, in the 2016 paper because of a study design complication: researchers randomly selected control villages, but did not collect baseline data in these villages. The lack of baseline data in the control group is not just a harmless omission, as in “we lose some power, no big deal.” Because there were eligibility criteria for receiving cash, but households were sampled a year later, no one can say for certain if the households sampled in the pure control villages at follow-up are representative of the would-be eligible households at baseline.

So, quite distressingly, we now have two choices to interpret the most recent findings:
  1. We either believe the integrity of the counterfactual group in the pure control villages, in which case the negative spillover effects are real, implying that total causal effects comparing treated and control villages are zero at best. Furthermore, there are no ITT effects on longer-term welfare of the beneficiaries themselves - other than an increase in the level of assets owned. In this scenario, it is harder to retain confidence in the earlier published impact findings that were based on within-village comparisons – although it is possible to believe that the negative spillovers are a longer-term phenomenon that truly did not exist at the nine-month follow-up.
  2. Or, we find the pure control sample suspect, in which case we have an individually randomized intervention and need to assume away spillover effects to believe the ITT estimates.
This is what is called a double-edged sword in English or the figurative Turkish stick with two shi*ty ends: your hand is going to get dirty no matter which end you hold it from…
 

Authors

Berk Özler

Lead Economist, Development Research Group, World Bank

Rasmus Schjoedt
March 26, 2018

This is what I was trying to get at here: cash transfers is a hype that is bound to fade away like all the other development fads that have come and gone. Life cycle social protection systems, on the other hand, are indisputably important for reducing poverty and inequality. We really need to move away from this obsession with limited poverty reduction interventions and focus on the broader transformation of societies and economies: http://www.developmentpathways.co.uk/resources/two-lives-social-protect…

Ashu Handa
March 26, 2018

While appreciating the limited space available in a blog, there are some important details left out of this post that readers should keep in mind. First, development partners and governments typically define a cash transfer as a regular and predictable monetary transfer to a particular target group. None of the studies (except possibly the Baird study) fit this description. All studies reported here are one-time lump-sum payments, usually tied to training or business start-up. None are poverty-targeted large-scale cash transfers of the type that are operated by national governments at scale as part of their social protection system (such as Mexico's Oportunidades, South Africa's Child Support Grant, or Zambia's Harmonized Social Cash Transfer Program). Those programs provide regular and predictable cash payments to specific target groups over long periods of time.
The Baird study is a CCT but had a UCT arm and was not poverty targeted. The Brudevold experiment had a UCT arm that was a one-time payment. All of these studies are experiments run by researchers or NGOs except possibly the Blattman study, but that was a lump-sum transfer conditional on business start-up. And Give Directly is an NGO that provides a lump-sum cash transfer provided in three installments--nothing at all like the Kenya Cash Transfer for Orphans and Vulnerable Children (CT-OVC), the largest cash transfer program in Kenya run by the government itself (and an order of magnitude larger in scale than Give Directly).
The fact that this blog focuses on NGO or researcher-run experiments also has important limitations for generalizability and what can be learned by governments. For a nice discussion see this WBRO paper http://tinyurl.com/y9y5ge46. So in fact, governments actually likely have very little to learn about their cash transfer programs based on this blog. First, the blog doesn't really talk about cash transfers in the traditional sense, none of the programs remotely resembles programs like the Kenya CT-OVC or the Malawi Social Cash Transfer Program--cash transfer programs governments are actually implementing. Second, the evidence is not from government programs, with important implications for what governments can take away. This makes the title of the blog particularly worrisome.
One interpretation of the evidence cited in this blog is that small lump-sum cash payments, particularly tied to entrepreneurship or business start-up, do not seem to have long lasting effects on poverty reduction. My sense is that we already knew that.

Berk Ozler
March 26, 2018

Thanks for the comment.
Many sentences here that are speculative or wrong, but I'll address them in my next post when I can do just a little more justice to the entire literature that you have contributed to. However, here is a sentence from the abstract of a review paper we just finished (that I cannot share just yet) regarding your last paragraph. The exact opposite of your sense of what we already knew:
"In contrast, transfers made for job search assistance or business start-up tend to increase adult labor supply and earnings, with the likely main channels being the alleviation of liquidity and risk constraints."

Marion Ouma
March 26, 2018

It is not only in the dissemination of evidence that researchers broadcast that which reinforces their position, but my research shows that promoters of cash transfers in Africa i.e. international organisations hide information from governments. For example in relation to Bolsa Famili, which a lot of the organisations pass on as the "programme to learn from", governments are not told that labour reforms contributed the greatest to poverty and inequality in Brazil. Instead cash transfer are hyped over labour reforms when the latter may the biggest game changer in poverty reduction. Broader reforms in social policy including access to education, health etc is where we need to go back in Africa instead of cash transfers. The cash hype may be bursting too soon.

Shahid Mehmood
March 26, 2018

Thank you Berk, for being honest in terms of this issue because (as this blog mentions) all you hear is accolades for cash transfers, but nobody discusses the other side. Given that i worked for about 2.5 years in my country's biggest cash transfer program (BISP, Pakistan), your's truly was witness to how billions of taxpayers money and one donated by donors was wasted and pilfered. In Pakistan, this large transfer program is usually billed as a 'success', yet nobody is willing to bring forth its darker side. For example, in 2015, the country's finance minister himself acknowledged that 127,000 fake beneficiaries had been receiving cash for many years. The cost: 7 billion rupees. But surprisingly, there is little if any discussion on this. Once, at a conference, when i mentioned this statistic to a DFID consultant who went out for the way to praise BISP, he immediately diverted the talk towards something unrelated. And that's simply because if these things are acknowledged, then many consultancies and jobs would be in danger. As this blog rightly points out, we want to hear only tat which suits our biases and interests.
I've always maintained that the biggest weakness of evaluating these programs is the opportunity cost to the economy and nation as a whole, and negative spillovers. In terms of BISP in Pakistan, for example, more than 3000 people were hired, and most were political appointees. Given lucrative consultancies and sky high pay packages, the governments and bureaucrats use it as a employment and hiring bureau rather than a social safety net program. That's just one darker aspect. If I mention more, the whole edifice would come crumbling down.
So, let's speak and write the truth, and keep our ears and brain open to all aspects.

Anthony Obeyesekere
March 26, 2018

Very interesting. Thanks. Looking forward to the next post.

Stephen Kidd
April 10, 2018

It's not at all surprising that the effects of short-term cash transfers wear off pretty quickly. Once people are no longer receiving cash, then any shock could set them back. We showed in our paper on the much-hyped Graduation schemes that impacts are minimal and don't last. See: http://bit.ly/2tDmDAd Yet, this doesn't stop CGAP promoting them as if they were the key to eliminating world poverty.
The point about inclusive, lifecycle social security schemes is that people are not taken off after a short period of time. Older people receive pensions until they die, children until they are 18 years of age, etc. If Ozler and Blattman were to receive $2,000 in a lump sum for their old age pension, I'm sure the effects wouldn't last for long. But, if they received $2,000 per month until they die, I'm sure that the impacts would be very significant and durable.
Best,
Stephen Kidd

majed el sayegh
May 01, 2021

when a poor in money has good mind, can invest the money help in100% advancing his technical job, and make on cultivation in his ground, today net is very help full by experimental knowledge.
this is essential today with limiting births, and strict open eye for the money given where its going