I received this email from one of our readers:
“I don't know as much about list experiments as I'd like. Specifically, I have a question about administering them and some of the blocking procedures. I read a few of the pieces you recently blogged about and have an idea for one of my own; however, here's what I'd like to know: when you send your interviewers or researchers out into the field to administer a list experiment, how do you ensure that they are randomly administering the control and treatment groups? (This applies to a developing country as opposed to a survey administered over the phone.) “
This question of how to randomize questions (or treatments) on the spot in the field is of course a much more general one. Here’s my reply:
I’ve done this in different ways in different places:
1) Using CAPI having the computer automatically randomize
2) After a listing survey, randomly assigning the IDs of the people we are then going to interview, and then having questionnaire booklets printed in two versions, with the IDs pre-written on them. Then the interviewer just pulls out the questionnaire booklet and won’t know until he/she gets to that question which version they are having.
3) Randomize this at the level of the interviewer (so the interviewers don’t know there is another version of the question), and then randomize which households interviewers visit (not my preferred option).
4) There is also the option of randomizing in the field itself (e.g. by having the interviewer toss a coin to figure out which version to use). I try to avoid this option, except for things like games with the respondent, where you have them roll a dice or toss a coin.
5) People also use things like “if their birthday is on an odd day, use version A, if on an even day, use version B” to allow this to be done in the field.
Anyone have any other recommendations, or a good recommendation for a reading source on how to randomize on the spot and how to monitor/check this well?
Join the Conversation