One of the things I learned in my first field work experience was that keeping interviews private was critical if you wanted unbiased information. Why? I guess at the time it should have been kind of obvious to me – there are certain questions that a person will answer differently depending on whom else is in the room. We were doing a socio-economic survey of rural households in Ghana, and we thought that income, in particular, would be sensitive, since spouses tended to share information on this selectively and perhaps in a strategic way.
So our protocol was that the interviews were to be conducted separately, by an enumerator of the same gender, and in private. As I spent more time in the field, going on interviews, I saw cases where this really mattered. My favorite case was an interview we were doing with a woman in one village. It was a hot day, so we were sitting outside under the tree and the enumerator was working through a set of questions on all of her loans – those that she made to others and those that she took. Her husband was circling around, at first casually walking by, but then coming closer in ever-narrowing circles, as he tried to listen in. Finally, the woman had had enough – she picked up the broom she used for sweeping and half rose out of her chair, waving at him and telling him to get lost. He took off. So clearly, she was telling us stuff she would really prefer he not hear. (Let’s not examine the effects of having a big sun-burned foreign guy sitting in on the interview).
On another interview, we were interviewing a couple that had been married for something like 40 years. When we were doing the expenditure module with the man, he kept going over to interrupt the wife (who was in the midst of her own interview) to get information. On one level this was annoying, since a goal of this expenditure module was to see how much they knew (without asking) about each other’s expenditure. On another level it was sweet. So for this household, it seemed like having the spouse present would result in better quality answers if we were trying to get a complete picture of expenditure.
In the end, there is likely to be a continuum of couples – some where they can answer better if the other is present, and others where they are better kept apart. Of course, what matters most is which questions are sensitive – with respect to the spouse, but also the kids, and neighbors and other household members. The obvious question topics to keep private are sex, violence, and money matters. But the complete list, in my experience varies a lot across different social contexts. On a recent survey pre-test in Ethiopia, we were working through some mobility questions and the one place this woman couldn’t go without her husband’s permission was the health center. He, on the other hand (separately), told us she could go anywhere she wanted. In other contexts, I am not sure their answers would be different, and in still others, I am not sure this would be a sensitive topic (in this particular case it turned out to be about birth control). Pre-tests will be useful for understanding this (but you have to beware the big sun-burned foreigner effect), as will be debriefing your enumerators and academics who come from and/or have worked in the area of study.
So how much does this matter for measurement? The strategic elements of private information are worthy of another (later) blog post, but a recent paper by my fellow blogger David McKenzie and coauthors has some interesting estimates with respect to (small) firm profits. They find that men and women under-report their profits by 8% (significantly so for men) when the spouse is present during the interview. This doesn’t affect their results, but a) it’s not a tiny number and, b) it is systematic bias, not classical measurement error.
So this is something to think about the next time you go out to pre-test a survey. Maybe it doesn’t matter for the particular question you are trying to answer – or maybe it does, and you (and I) don’t know it yet. The truth is out there; let’s see if we can find it.
Join the Conversation