Concerns about external validity are a common critique of micro work in development, especially experimental work. While not denying that it is useful to learn what works in a variety of different settings, there seems to be two forms of double-standard (or a double double-standard) going on: first, economic journals and economists in general seem to apply it to work on developing countries more than they do to other forms of research; and second, this concern seems to be expressed about experiments more than other micro work in development.
· Chris Blattman on a new paper by David Card, Stefano DellaVigna, and Ulrike Malmendier on how many experiments are getting published in top economics journals, and on the role of theory in these.
Abhijit and Esther very graciously agreed to reply to Monday’s review of their book. Here is their reply, followed by a couple of additional thoughts from me.
Markus’ s post yesterday is the first on what will be one recurring blog theme here- measurement. I’ll continue the trend today with a focus on one of the most fundamental welfare constructs in economics: consumption. Specifically, how might the development researcher accurately measure household consumption through survey?
One of the things I learned in my first field work experience was that keeping interviews private was critical if you wanted unbiased information. Why? I guess at the time it should have been kind of obvious to me – there are certain questions that a person will answer differently depending on whom else is in the room. We were doing a socio-economic survey of rural households in Ghana, and we thought that income, in particular, would be sensitive, since spouses tended to share information on this selectively and perhaps in a strategic way.
Hot on the heels of More than good Intentions comes an outstanding new book by two of the most prominent leaders of the recent push for more rigorous evaluation – Abhijit Banerjee and Esther Duflo’s Poor Economics: A Radical Rethinking of the way to Fight Global Poverty
I attended this conference at Madison, WI last week, which was quite pleasant except the weather – it snowed!
As a PhD student in the late 90s, randomized field trials were not yet common place in empirical development economics. Certain quasi-experimental methods such as regression discontinuity were also fairly exotic. It was the era of the “natural experiment”, when fellow PhD students scoured county newspapers at the university library for research leads. These students were looking for news of policy changes that might plausibly introduce some exogenous variation in the local market environment.
As a fair number of impact evaluations I work on are programs designed by governments or NGOs, I often initially have to have a tricky discussion when it comes time to do the power calculations to design the impact evaluation. The subject of this conversation is the anticipated effect size. This is a key parameter – if it’s too optimistic you run the risk of an impact evaluation with no effect even when the program had worked to some (lesser) degree, if it’s too pessimistic, then you are wasting money and people’s time in your survey.
Millions of dollars are spent each year trying to improve the productivity of firms in Africa (and those in other developing countries), yet we have very little rigorous evidence as to what works. In a new working paper I look at whether it is even possible to learn whether such policies even work, and what can be done to make progress.
Small number of firms + Large heterogeneity = Not much power