Syndicate content

pre-analysis plans

Preregistration of studies to avoid fishing and allow transparent discovery

Berk Ozler's picture
The demand for pre-analysis plans that are registered at a public site prior available for all consumers to be able to examine has recently increased in social sciences, leading to the establishment of several social science registries. David recently included a link to Ben Olken’s JEP paper on pre-analysis plans in Economics. I recently came across a paper by Humphreys, de la Sierra, and van der Windt (HSW hereon) that proposes a comprehensive nonbinding registration of research. The authors end up agreeing on a number of issues with Ben, but still end up favoring a very detailed pre-analysis plan. As they also report on a mock reporting exercise and I am also in the midst of writing a paper that utilized a pre-analysis plan struggling with some of the difficulties identified in this paper, I thought I’d link to it a quickly summarize it before ending the post with a few of my own thoughts.

The Impact of Vocational Training for the Unemployed in Turkey: an inside look at my latest paper

David McKenzie's picture
My latest working paper (joint with Sarojini Hirschleifer, Rita Almeida and Cristobal Ridao-Cano) presents results from an impact evaluation of a large-scale vocational training program for the unemployed in Turkey. I thought I’d briefly summarize the study, and then discuss a few aspects that may be of more general interest.

The study

Creativity vs. fishing for results in scientific research

Berk Ozler's picture
One of my favorite bloggers, Andrew Gelman, has a piece in Slate.com in which he uses a psychology paper that purported to show women are more likely to wear red or pink when they are most fertile as an example of the ‘scientific mass production of spurious statistical significance.’ Here is an excerpt:

Does Angus Deaton worry too much about wolves eating his t-values?

David McKenzie's picture

In his latest Letter from America in the Royal Economic Society’s newsletter, Angus Deaton says “your wolf is interfering with my t-value” (the title refers in part to regulations on hunting wolves in the American West) and talks about excessive regulation with NIH grants, and his concerns with the move towards trial registries:

A pre-analysis plan checklist

David McKenzie's picture

A pre-analysis plan is a step-by-step plan setting out how a researcher will analyze data which is written in advance of them seeing this data (and ideally before collecting it in cases where the researcher is collecting the data). They are recently starting to become popular in the context of randomized experiments, with Casey et al. and Finkelstein et al.’s recent papers in the QJE both using them.