Syndicate content

John Henry effect

What does a game-theoretic model with belief-dependent preferences teach us about how to randomize?

David McKenzie's picture

The June 2017 issue of the Economic Journal has a paper entitled “Assignment procedure biases in randomized policy experiments” (ungated version). The abstract summarizes the claim of the paper:
“We analyse theoretically encouragement and resentful demoralisation in RCTs and show that these might be rooted in the same behavioural trait –people’s propensity to act reciprocally. When people are motivated by reciprocity, the choice of assignment procedure influences the RCTs’ findings. We show that even credible and explicit randomisation procedures do not guarantee an unbiased prediction of the impact of policy interventions; however, they minimise any bias relative to other less transparent assignment procedures.”

Of particular interest to our readers might be the conclusion “Finally, we have shown that the assignment procedure bias is minimised by public randomisation. If possible, public lotteries should be used to allocated subjects into the two groups”

Given this recommendation, I thought it worth discussing how they get to this conclusion, and whether I agree that public randomization will minimize such bias.

Are John Henry Effects as Apocryphal as their Eponym?

David McKenzie's picture

Many people are aware of the concept of a “placebo effect” in medicine and of the idea of a Hawthorne effect – in both cases the concern is that merely being treated can cause the treatment group in an intervention to change their behavior, regardless of what the treatment actually is.