This blog has now featured a healthy debate between researchers advocating randomized evaluations and those cautioning the overuse of such methods. One point that I believe both sides would agree on is that irrespective of which empirical methods we use, it is important to understand and analyze the causal chain of impact. Such analysis can greatly enhance the external validity of any evaluation.
In a psychology experiment from 15 years ago, participants were asked to remember a number – the number was randomly selected to either be a short two digit number or a seven digit number – and then to walk down a hallway to another room for an interview. As a seeming afterthought, they were told there is a snack cart in the hallway and to help themselves to one of the snacks. The snack choice was either fruit salad or chocolate cake.
Berk (who is on vacation this week) and I have recently been surveying assistant professors, graduate students, and World Bank economists to learn how they find out about new research and the role of blogs in this process. We’ll be sharing results once we have finished this, but to start with, I thought I’d share this chart below on what junior faculty who work on development think are the under-researched topics.
- The big picture
So recently one of the government agencies I am working with was telling me that they were getting a lot of pressure from communities who had been randomized out of the first phase of a program. The second phase is indeed coming (when they will get the funding for their phase of the project) but the second round of the survey has been delayed – as was implementation of the first round of the program. But that doesn’t make the pressure any less understandable.
A number of recent field experiments have been conducted within firms and across firms. In another paper in what is shaping up to be an excellent forthcoming Journal of Economic Perspectives symposia on experiments, Oriana Bandiera,Iwan Barankay and Imran Rasul give their take of what we have learned from firm experiments so far, and their ideas on further research directions.
Field experiments within firms
- Financial Sector
Numerous recent discussions on the future of development financing focus on the delivery of results and how to mainstream accounting for results in aid flows (see here for one review paper by Nemat Shafik). This “results based approach” to aid is gathering steam in many contexts.
Last week I wrote about “treatment as prevention.” Because being treated by a combination of ARV drugs effectively prevents the transmission of HIV from an infected person to his (her) uninfected partner, the idea is that if we were to test as many people as possible, find out who is infected, and offer them ARVs, we could make significant headway in preventing the spread of HIV. In other words, test and treat.
So in my quest to understand the gender dimensions of water supply this week, I stumbled upon a nice paper by Florencia Devoto and coauthors. They look at the effects of providing piped water in Tangiers, Morocco. The immediately cool thing about this paper is that they got something quite hard – randomization in an infrastructure project.
In an interesting new paper, Jens Ludwig, Jeffrey Kling and Sendhil Mullainathan argue that economists should be doing more experiments to identify behavioral mechanisms, and that these can be central to policy, even if the experiments themselves are far from what a policymaker would implement. So what are these mechanism experiments, and what can we learn from them?