Syndicate content

Add new comment

Submitted by Jenny Aker on

David,

Thanks to you (and Chris B) for blogging about this. I agree that the evaluators have designed a nice experiment, and done a great job thinking through some of these important questions related to UCTs. I have three additional thoughts about this:

1. How should we think about these spillover effects? You mention that they don't find evidence of negative spillovers. I might have read this too quickly, but while it's true that a majority of the variables don't show negative spillover effects, I do think that the spillover effects merit further attention by the authors. There are several cases of negative and statistically significant spillover effects - on the value of particular assets, social expenditures, livestock revenues. Now whether these effects are due to multiple hypothesis testing or true negative spillovers is unclear, but I think that they merit further attention -- especially as several of them are related to control households' asset values, which suggest that perhaps there are some general equilibrium effects that the village price analysis isn't capturing. (In addition, for some of them, the magnitude is quite large - ie, as large as the treatment effect, unless I'm missing something). This is particularly important in an intervention such as this one, where eligible households weren't chosen for the treatment (and the lottery wasn't done publicly, although it was announced).

2. Second, are village prices the right level to be thinking about inflationary effects of cash transfers? I know that many impact evaluations often focus on these, but in many rural areas of sub-Saharan Africa, most purchases occur outside of the village at weekly agricultural markets. While village-level prices are important, and might capture one aspect of inflationary effects of cash transfers, it seems as if a relevant level of impact is on prices at the market level (which might or might not be integrated with village prices).

3. My third point is related to all impact evaluations, but something I think it's important to keep in mind. More often than not, our impact evaluations don't address the primary issue: Was this the right intervention for the problem in the first place? Yes, UCTs have worked in this context, but compared to what? (Getting back to the NPR piece, would Heifer International's intervention have done better, worse or the same in addressing poverty here)? Obviously that wasn't the objective or focus of this impact evaluation, but I am often surprised by how often we assume that we've chosen the right intervention for the problem (and any evidence of impact is proof of this). Finding that cash transfers had an impact isn't proof that cash transfers were the right intervention in the first place. Obviously this is difficult issue to resolve, but I would love for us all to have more discussion on how we identify and analyze the problem and design interventions in the first place, even before we get to testing whether (and why) they worked.