How do cash transfers stack up against a psychotherapy program in terms of not only economic outcomes, but mental health? That’s the subject tackled by an interesting recent paper by Johannes Haushofer, Robert Mudida, and Jeremy Shapiro. And there is some methodological coolness to boot. Headline result: cash wins, for all outcomes.
It’s off to rural Kenya, where Haushofer and co. go to a bunch of villages and screen households for poverty by building conditions. Selected beneficiaries are randomized into three different treatment conditions. First off is cash (CT). This is a large, unconditional cash transfer of $485 (1076 adjusted for purchasing power parity), which is about 20 months of per capita consumption.
The second intervention is a psychotherapy intervention called problem management plus (PM+). It was developed by the WHO and adapted to the Kenyan context. It’s five ninety minutes sessions delivered by community health workers. The goal is to help individuals engage in solving problems and managing their stress. And it’s been shown in a previous study (working with women around IPV) to be effective in improving mental health. A third group got both the CT and PM+
There are a bunch of bells and whistles here. Haushofer and co. randomize within village within clusters of households to look for spillover effects (there are none of note). They also randomize some cash transfer folks to weekly transfer (once a week for 5 weeks) or one lump sum transfer.
Take up and compliance with the interventions was quite good and Haushofer and co. had an impressive set of quality controls on the psychotherapy delivery (plus it was delivered by the same NGO as in the study before) as well as calling folks to make sure they got their cash transfer.
A year later, the cash has impacts across the board. Monthly per capita consumption is up by around $10 (20%), assets are $272 higher (49%), and household revenue is up by $36 per month (27%). And cash transfers boost a psychological well being index by .24 standard deviations. This index includes a screener for psychiatric conditions (GHQ-12), a perceived stress scale, and happiness and life satisfaction questions. The cash had significant positive impacts on all of these. The PM+ on the other hand, has no impact on any of these outcomes, and a significant positive effect (not good) on a domestic violence index. This last result doesn’t survive multiple hypothesis correction but is still something to keep an eye on.
When you give folks both interventions at once, you don’t get anything more than either alone. Indeed, you might get less consumption than cash alone (significant at 10 percent).
Exploring the cash in more detail, it turns out that the weekly transfers had significantly higher impacts on consumption and revenues.
And, in case you were wondering, the cash transfer is markedly cheaper than the psychotherapy intervention.
Ok, now for the methodological coolness.
The first issue is that Haushofer and co. have a null result. And a null result for a program that previously showed a positive impact. As anyone who has had one of these knows, it’s a lot more work than a positive result. Haushofer and co. do a nice job of walking us through a bunch of different ways to think about this. First, they take us through the nitty gritty of implementation. It turns out that this version was generalized while the previous version focused on GBV. The previous implementation went to urban women, this version went to rural men and women (though there are no positive effects for women when Haushofer and co. do some heterogeneity work). The other study measured results at three months post intervention, Haushofer and co. are measuring things a year later (when Haushofer and co. look at variation (not including three months) in time since treatment, they find nothing. Haushofer and co. also note that there have been similar programs elsewhere that show significant impacts and are significantly more intensive than these two in Kenya. So, putting it all together they aren’t ready to give up on a more focused and intensive version of this kind of psychotherapy.
Getting back to their null result, Haushofer and co. then take us through a Bayesian approach to give us a perspective on what the likelihood of different effect sizes are. Finally, back to implementation. The community health workers were assigned randomly. So, Haushofer and co. estimate effects by worker (and throw in some out-of-sample tests) and, indeed, there are some who deliver the intervention in a way that suggests large impacts. Taken together, this is a good guide about how to think about and document a null result – and maybe draw some positive lessons for policy.
The second dimension of methodological coolness is a test they run for desirability bias in responsiveness. Maybe respondents are telling them what they think they want to hear (OK, this pattern of results would take some wacky respondents to have desirability bias give us these results, but its still a cool test). In the control group they randomize folks across positive demand prompts and negative demand prompts. These prompts give the respondent a signal on what the researchers are thinking. For example: “we hypothesize that people who participated in this study and received the same treatment as you will give higher [lower] responses to these questions than others,” followed by the outcome. They find no difference in responses. This approach is based on an earlier paper, which has more on the methods to complement the pretty comprehensive discussion in the current paper.
So this is a neat paper – showing us the multi-dimensional benefits of cash transfers. I’m particularly keen to understand the mechanisms here – both within the economic effects but also how the psychological outcomes are linked (e.g. which comes first? Does it matter?).
Join the Conversation