A pre-analysis plan is a step-by-step plan setting out how a researcher will analyze data which is written in advance of them seeing this data (and ideally before collecting it in cases where the researcher is collecting the data). They are recently starting to become popular in the context of randomized experiments, with Casey et al. and Finkelstein et al.’s recent papers in the QJE both using them.
As our impact evaluations broaden to consider more and more possible outcomes of economic interventions (an extreme example being the 334 unique outcome variables considered by Casey et al. in their CDD evaluation) and increasingly investigate the channels of impact through subgroup heterogeneity analysis, the issue of multiple hypothesis testing is gaining increasing prominence.
Given the massive debate in the U.S. about government health insurance, the just released results of a new experiment are justly making headlines. In 2004, the state of Oregon, due to budgetary shortfalls, closed its public health insurance program for low-income people. In early 2008, the state decided it had enough budget to fund 10,000 new spots. Given that it expected demand for these new slots to far exceed supply, the state Government opened up a sign-up window, getting 90,000 people to sign-up for a waitlist, and then used random lottery draws to select people from the waitlist.
