The latest issue of the Journal of Economic Perspectives (all content openly available online), has a symposium on the use of field experiments in economics. We’ve discussed or linked to posts on three of the four papers in previous blog posts: A paper on mechanism experiments by Ludwig, Kling and Mullainathan; a paper on the role of theory in experiments by Card, DellaVigna and Malmendier; and a paper summarizing recent work on firm experiments by Bandiera, Barankay and Rasul. The fourth paper is an introductory piece by John List entitled “why economists should conduct field experiments and 14 tips for pulling one off”. This is a very nice intro, and as well as useful suggestions for those of you thinking about doing an experiment, some points which I thought worth noting:
· A little section on “What happened to informed consent?” – List argues that “In experiments that can affect the physical health of the participants, voluntary consent remains essential. However, the case for voluntary consent in economic experiments is less clear-cut.”. He notes that in many economic experiments, the participants suffer no physical harm from participating, and in many cases, may not ever realize they were in an experiment at all. Since Hawthorne effects (behavior change when people know they are being observed) are likely important “there are valid arguments for not making informed consent an ironclad rule in natural field experiments”
· On the different skills required for doing experiments: “conducting successful field experiments demands a different set of skills from traditional economic research, including the ability to recognize opportunities for experimentation hidden amidst everyday phenomena, an understanding of experimental design, and the interpersonal skills to manage what are often a complex set of relationships involving parties to an experiment.”
· On concerns about fairness: “The line of argument is that it is not fair to only give a fraction of the population a potentially beneficial treatment. While I am sympathetic to this line of reasoning, it is ultimately flawed. First of all, it only considers contemporaneous trade-offs. One could easily argue that it is not fair to future generations to bypass learning opportunities that could make them better off. I am personally glad that earlier generations executed experiments to determine the efficacy of promising drugs so that today my father’s heart condition can be treated appropriately. Second, even if one insists on everyone receiving treatment, it remains possible to execute an experiment whereby people receive treatment in waves over time.”
This intertemporal fairness argument is phrased more elegantly here than I’ve seen it phrased before, and the other advice he has such as “Run the field experiment yesterday rather than tomorrow” are also valuable.
And for something completely different, the same issue also has a symposium on migration. I have a paper with John Gibson that looks at the impact of brain drain through answering 8 questions about brain drain.
Join the Conversation