Last year, Banerjee and coauthors published a paper in Science that showed the striking impacts of poverty graduation programs in 6 countries after three years. This week, we get a new paper from Bandiera and coauthors that revisits one of the models of this type of program they wrote about in 2013 and looks not only at a wide range of benefits, but also at what happens in the longer run.
First, a quick recap on the program. Bandiera and co. are looking at the Targeting the Ultra-Poor (TUP) program implemented by the NGO BRAC in Bangladesh. This is a bundle of interventions. The most expensive parts are the transfer of a large asset (mostly cows) plus training and support on how to care for them. But there are also a bunch of cheaper interventions including health support, training on legal and social rights, and encouragement to save and engage with BRACs microfinance program. So this is dealing with more than one constraint at a time. These benefits are given to the folks in the village defined as ultra-poor through a participatory village ranking exercise.
Bandiera and co. have a mass of data – following (randomly assigned) treatment and control groups from baseline in 2007 through follow-ups in 2009 and 2011. And in this version of the paper, they give us some sense of what happened through 2014. Before turning to these longer term effects, let’s take a closer look at what happens after four years (before some of the controls get treated and about 2 years after the program wrapped up all main activities).
Bandiera and co. break out the results nicely in three different ways. The give us not only the percentage point gain (ppt) or absolute increase for each dimension, but also the percent gain over baseline and the distribution of impacts using quantile treatment effects (QTE). I won’t talk about the last in all cases, but in some it shows some interesting heterogeneity.
Labor supply. Keep in mind that this program targets women. And women in these villages spend most of their time in three activities: agricultural wage work, working as a maid, or rearing livestock. If you’re ultra-poor, it’s primarily the first two (the last requires capital and some different skills). The program shifts this, increasing ultra-poor women’s annual hours in livestock by 415 or 316%. Hours spent being a maid fall by 117, or 36%. Overall, there is a significant increase in the labor supply of ultra-poor women.
Earnings and consumption. Earnings from livestock rearing goes up from US$80 after 2 years to $115 after 4 years. In line with the reduced supply of labor into maid labor and agricultural activities, wage rates for these go up in the treated villages (no Lewis-like surplus labor here!) Overall, for those treated, annual earnings are up 37% relative to baseline. Bandiera and co take the critical extra step here: measuring household consumption (not trivial) and then benchmarking this against a poverty line of $1.25 a day. They find that poverty falls by 8.4 ppt or 15% -- a sizable impact. Bandiera and co. also look at this effect across the distribution and find that the consumption benefits are significantly larger for higher centiles than for the lower ones.
Assets and savings. Household assets are up by 110%, with a significantly larger effect at four years than at two. Savings (in an institution) are also up – by US$53 or nine times baseline levels. Beneficiary households are also more likely to have a loan (11 ppt or 61%) and more likely to be making loans to others (5 ppt or 464%). In terms of productive assets, after four years, the (real) value of the cows is 16% more than the asset transfer – so some significant growth in the asset after the program stopped. Ultra-poor households are more likely to be renting land (11 ppt or 190%) and owning land (2.6 ppt or 38%), with the value of land holdings also up (US$327). The land results are particularly encouraging since land is not a focus of the program and landlessness is highly correlated with poverty. Business assets are also up (283%). As with consumption, the effects for assets as a whole are much higher for the households at the higher centiles, and much lower for those at the bottom of the distribution.
Given these rather significant effects, Bandiera and co. also look to see how the program might have shifted things for others in the village. For starters, keep in mind that the program is targeting, on average, 6% of the village as direct beneficiaries. Turning to livestock values or labor for non-ultra-poor women, Bandiera and co. find no effects. In non-livestock work (agricultural labor and work as maids), recall that the program seems to have increased wages for this. So the ineligible women who work in this sector benefit from this dimension. There is a decline in hours worked in this amongst ineligible women, but it’s not significant (nor is the effect on total earnings). Other dimensions also show no significant effects (with the exception of an increase in business assets of the ineligible women – but these are not a large part of the asset pool).
Bandiera and co. then turn to looking at how this might have shifted the within village income/wealth distribution. They find that the ultra-poor close the gap between themselves and the near-poor (the next group up the ladder) in terms of consumption and household assets, while passing them in terms of savings and productive assets (where the ultra-poor now hold twice the value of the near-poor). So there is significant movement in the village income distribution here.
So what happens after seven years? In terms of methodology, this is a section worth thinking about since the profusion of impact evaluations is now reaching a maturity where we can start asking this kind of question of more of our programs. Bandiera and co. face a problem that I’ve seen in a number of the programs where I would like to look at longer term effects: contamination of the control group by design. At seven years, 49% of the control has been treated, in part because of the early success. So what can you do when the program scales-up (for good operational reasons, based on evidence)?
Bandiera and co offer us two ways to look at these potential longer term effects: 1) by mapping out the progression of the treatment group from year four to year seven without looking at the control, and 2) by using the QTE estimates (at the median, 25th and 75th percentiles) of the original treatment to create different (plausible) counterfactuals of what happens to the treated controls. The first approach gives us a strong suggestion that effects may have increased over the longer term (with the exception of savings which takes a dip). The second approach gives us solid evidence of program sustainability, barring some major reversal in previous patterns.
So adding to the earlier results we’ve seen on this kind of program, these results are exciting. They indicate a sustainable (and cost effective – see the paper for the numbers) way to break poverty traps and map a clear and sustainable trajectory out of poverty. And along the way it opens up a host of super interesting research questions: understanding the heterogeneity of benefits across the distribution, understanding whether it’s the whole bundle of interventions at once that matters or some (cheaper) part, and seeing whether the effects will be different if we target men instead of women.
First, a quick recap on the program. Bandiera and co. are looking at the Targeting the Ultra-Poor (TUP) program implemented by the NGO BRAC in Bangladesh. This is a bundle of interventions. The most expensive parts are the transfer of a large asset (mostly cows) plus training and support on how to care for them. But there are also a bunch of cheaper interventions including health support, training on legal and social rights, and encouragement to save and engage with BRACs microfinance program. So this is dealing with more than one constraint at a time. These benefits are given to the folks in the village defined as ultra-poor through a participatory village ranking exercise.
Bandiera and co. have a mass of data – following (randomly assigned) treatment and control groups from baseline in 2007 through follow-ups in 2009 and 2011. And in this version of the paper, they give us some sense of what happened through 2014. Before turning to these longer term effects, let’s take a closer look at what happens after four years (before some of the controls get treated and about 2 years after the program wrapped up all main activities).
Bandiera and co. break out the results nicely in three different ways. The give us not only the percentage point gain (ppt) or absolute increase for each dimension, but also the percent gain over baseline and the distribution of impacts using quantile treatment effects (QTE). I won’t talk about the last in all cases, but in some it shows some interesting heterogeneity.
Labor supply. Keep in mind that this program targets women. And women in these villages spend most of their time in three activities: agricultural wage work, working as a maid, or rearing livestock. If you’re ultra-poor, it’s primarily the first two (the last requires capital and some different skills). The program shifts this, increasing ultra-poor women’s annual hours in livestock by 415 or 316%. Hours spent being a maid fall by 117, or 36%. Overall, there is a significant increase in the labor supply of ultra-poor women.
Earnings and consumption. Earnings from livestock rearing goes up from US$80 after 2 years to $115 after 4 years. In line with the reduced supply of labor into maid labor and agricultural activities, wage rates for these go up in the treated villages (no Lewis-like surplus labor here!) Overall, for those treated, annual earnings are up 37% relative to baseline. Bandiera and co take the critical extra step here: measuring household consumption (not trivial) and then benchmarking this against a poverty line of $1.25 a day. They find that poverty falls by 8.4 ppt or 15% -- a sizable impact. Bandiera and co. also look at this effect across the distribution and find that the consumption benefits are significantly larger for higher centiles than for the lower ones.
Assets and savings. Household assets are up by 110%, with a significantly larger effect at four years than at two. Savings (in an institution) are also up – by US$53 or nine times baseline levels. Beneficiary households are also more likely to have a loan (11 ppt or 61%) and more likely to be making loans to others (5 ppt or 464%). In terms of productive assets, after four years, the (real) value of the cows is 16% more than the asset transfer – so some significant growth in the asset after the program stopped. Ultra-poor households are more likely to be renting land (11 ppt or 190%) and owning land (2.6 ppt or 38%), with the value of land holdings also up (US$327). The land results are particularly encouraging since land is not a focus of the program and landlessness is highly correlated with poverty. Business assets are also up (283%). As with consumption, the effects for assets as a whole are much higher for the households at the higher centiles, and much lower for those at the bottom of the distribution.
Given these rather significant effects, Bandiera and co. also look to see how the program might have shifted things for others in the village. For starters, keep in mind that the program is targeting, on average, 6% of the village as direct beneficiaries. Turning to livestock values or labor for non-ultra-poor women, Bandiera and co. find no effects. In non-livestock work (agricultural labor and work as maids), recall that the program seems to have increased wages for this. So the ineligible women who work in this sector benefit from this dimension. There is a decline in hours worked in this amongst ineligible women, but it’s not significant (nor is the effect on total earnings). Other dimensions also show no significant effects (with the exception of an increase in business assets of the ineligible women – but these are not a large part of the asset pool).
Bandiera and co. then turn to looking at how this might have shifted the within village income/wealth distribution. They find that the ultra-poor close the gap between themselves and the near-poor (the next group up the ladder) in terms of consumption and household assets, while passing them in terms of savings and productive assets (where the ultra-poor now hold twice the value of the near-poor). So there is significant movement in the village income distribution here.
So what happens after seven years? In terms of methodology, this is a section worth thinking about since the profusion of impact evaluations is now reaching a maturity where we can start asking this kind of question of more of our programs. Bandiera and co. face a problem that I’ve seen in a number of the programs where I would like to look at longer term effects: contamination of the control group by design. At seven years, 49% of the control has been treated, in part because of the early success. So what can you do when the program scales-up (for good operational reasons, based on evidence)?
Bandiera and co offer us two ways to look at these potential longer term effects: 1) by mapping out the progression of the treatment group from year four to year seven without looking at the control, and 2) by using the QTE estimates (at the median, 25th and 75th percentiles) of the original treatment to create different (plausible) counterfactuals of what happens to the treated controls. The first approach gives us a strong suggestion that effects may have increased over the longer term (with the exception of savings which takes a dip). The second approach gives us solid evidence of program sustainability, barring some major reversal in previous patterns.
So adding to the earlier results we’ve seen on this kind of program, these results are exciting. They indicate a sustainable (and cost effective – see the paper for the numbers) way to break poverty traps and map a clear and sustainable trajectory out of poverty. And along the way it opens up a host of super interesting research questions: understanding the heterogeneity of benefits across the distribution, understanding whether it’s the whole bundle of interventions at once that matters or some (cheaper) part, and seeing whether the effects will be different if we target men instead of women.
Join the Conversation