If one wants to grow an oak tree, it helps to have both an acorn and a working knowledge of the conditions under which an acorn is most likely to become an oak tree. One also needs to know how long the germination process is likely to take – in the case of the red oak, upwards of two years from flowering to acorn to sapling. Absent such knowledge, one might reasonably (but incorrectly) infer that, upon seeing no outward signs of life six months after planting the acorn, one’s efforts had been in vain.
Following on David’s rant on external validity yesterday, which turned out to be quite popular, I decided to keep the thread going. Despite the fact that the debate is painted in ‘either/or’ terms, my feeling is that there are things that careful researchers/evaluators can do to improve the external validity of their studies.
Concerns about external validity are a common critique of micro work in development, especially experimental work. While not denying that it is useful to learn what works in a variety of different settings, there seems to be two forms of double-standard (or a double double-standard) going on: first, economic journals and economists in general seem to apply it to work on developing countries more than they do to other forms of research; and second, this concern seems to be expressed about experiments more than other micro work in development.
· Chris Blattman on a new paper by David Card, Stefano DellaVigna, and Ulrike Malmendier on how many experiments are getting published in top economics journals, and on the role of theory in these.
Abhijit and Esther very graciously agreed to reply to Monday’s review of their book. Here is their reply, followed by a couple of additional thoughts from me.
Markus’ s post yesterday is the first on what will be one recurring blog theme here- measurement. I’ll continue the trend today with a focus on one of the most fundamental welfare constructs in economics: consumption. Specifically, how might the development researcher accurately measure household consumption through survey?
One of the things I learned in my first field work experience was that keeping interviews private was critical if you wanted unbiased information. Why? I guess at the time it should have been kind of obvious to me – there are certain questions that a person will answer differently depending on whom else is in the room. We were doing a socio-economic survey of rural households in Ghana, and we thought that income, in particular, would be sensitive, since spouses tended to share information on this selectively and perhaps in a strategic way.
Hot on the heels of More than good Intentions comes an outstanding new book by two of the most prominent leaders of the recent push for more rigorous evaluation – Abhijit Banerjee and Esther Duflo’s Poor Economics: A Radical Rethinking of the way to Fight Global Poverty