In his latest Letter from America in the Royal Economic Society’s newsletter, Angus Deaton says “your wolf is interfering with my t-value” (the title refers in part to regulations on hunting wolves in the American West) and talks about excessive regulation with NIH grants, and his concerns with the move towards trial registries:
A pre-analysis plan is a step-by-step plan setting out how a researcher will analyze data which is written in advance of them seeing this data (and ideally before collecting it in cases where the researcher is collecting the data). They are recently starting to become popular in the context of randomized experiments, with Casey et al. and Finkelstein et al.’s recent papers in the QJE both using them.
If you are like most people working with quantitative data in development, getting too many statistically significant results is probably not your most pressing problem. On the contrary, if you are lucky enough to find a star, whether it's of the 1%, 5% or 10% type, there are plenty of star-killers to choose from. In what is perhaps the only contribution to the rare genre of 'econometrics haiku', Keisuke Hirano reflects on one of them: T-stat looks too good // Try clustered standard errors - // Significance gone (in Angrist and Pischke's MHE).
