Syndicate content

randomization methods

What does a game-theoretic model with belief-dependent preferences teach us about how to randomize?

David McKenzie's picture

The June 2017 issue of the Economic Journal has a paper entitled “Assignment procedure biases in randomized policy experiments” (ungated version). The abstract summarizes the claim of the paper:
“We analyse theoretically encouragement and resentful demoralisation in RCTs and show that these might be rooted in the same behavioural trait –people’s propensity to act reciprocally. When people are motivated by reciprocity, the choice of assignment procedure influences the RCTs’ findings. We show that even credible and explicit randomisation procedures do not guarantee an unbiased prediction of the impact of policy interventions; however, they minimise any bias relative to other less transparent assignment procedures.”

Of particular interest to our readers might be the conclusion “Finally, we have shown that the assignment procedure bias is minimised by public randomisation. If possible, public lotteries should be used to allocated subjects into the two groups”

Given this recommendation, I thought it worth discussing how they get to this conclusion, and whether I agree that public randomization will minimize such bias.

Should we require balance t-tests of baseline observables in randomized experiments?

David McKenzie's picture

I received an email recently from a major funder of impact evaluations who wanted my advice on the following question regarding testing baseline balance in randomized experiments:

Should we continue to ask our grantees to do t-tests and f-tests to assess the differences in the variables in the balance tables during the baseline?  

Is optimization just re-randomization redux? Thoughts on the recent ‘don’t randomize, optimize’ papers.

David McKenzie's picture
A couple of weeks ago, Berk blogged about a new paper by Bertsimas, Johnson and Kallus which argues that instead of randomization, it can be superior for power to choose the best of all possible allocations of subjects to treatment and control, where best is defined in terms of minimizing discrepancies in the mean and variance of the two groups.

From my email correspondence: how to randomize in the field

David McKenzie's picture

I received this email from one of our readers:
“I don't know as much about list experiments as I'd like.  Specifically, I have a question about administering them and some of the blocking procedures.  I read a few of the pieces you recently blogged about and have an idea for one of my own; however, here's what I'd like to know: when you send your interviewers or researchers out into the field to administer a list experiment, how do you ensure that they are randomly administering the control and treatment groups? (This applies to a developing country as opposed to a survey administered over the phone.) “
This question of how to randomize questions (or treatments) on the spot in the field is of course a much more general one. Here’s my reply: