Following on David’s rant on external validity yesterday, which turned out to be quite popular, I decided to keep the thread going. Despite the fact that the debate is painted in ‘either/or’ terms, my feeling is that there are things that careful researchers/evaluators can do to improve the external validity of their studies.
Concerns about external validity are a common critique of micro work in development, especially experimental work. While not denying that it is useful to learn what works in a variety of different settings, there seems to be two forms of double-standard (or a double double-standard) going on: first, economic journals and economists in general seem to apply it to work on developing countries more than they do to other forms of research; and second, this concern seems to be expressed about experiments more than other micro work in development.
· Chris Blattman on a new paper by David Card, Stefano DellaVigna, and Ulrike Malmendier on how many experiments are getting published in top economics journals, and on the role of theory in these.
Abhijit and Esther very graciously agreed to reply to Monday’s review of their book. Here is their reply, followed by a couple of additional thoughts from me.
Markus’ s post yesterday is the first on what will be one recurring blog theme here- measurement. I’ll continue the trend today with a focus on one of the most fundamental welfare constructs in economics: consumption. Specifically, how might the development researcher accurately measure household consumption through survey?
One of the things I learned in my first field work experience was that keeping interviews private was critical if you wanted unbiased information. Why? I guess at the time it should have been kind of obvious to me – there are certain questions that a person will answer differently depending on whom else is in the room. We were doing a socio-economic survey of rural households in Ghana, and we thought that income, in particular, would be sensitive, since spouses tended to share information on this selectively and perhaps in a strategic way.
Hot on the heels of More than good Intentions comes an outstanding new book by two of the most prominent leaders of the recent push for more rigorous evaluation – Abhijit Banerjee and Esther Duflo’s Poor Economics: A Radical Rethinking of the way to Fight Global Poverty
I attended this conference at Madison, WI last week, which was quite pleasant except the weather – it snowed!
As a PhD student in the late 90s, randomized field trials were not yet common place in empirical development economics. Certain quasi-experimental methods such as regression discontinuity were also fairly exotic. It was the era of the “natural experiment”, when fellow PhD students scoured county newspapers at the university library for research leads. These students were looking for news of policy changes that might plausibly introduce some exogenous variation in the local market environment.