Markus’s previous post on the measurement of sensitive information has started the ball rolling on a major topic that we all confront in field work – accurate measurement. This is an especially acute issue for studies that investigate socially undesirable or stigmatized behaviors such as risky sexual practices or illegal activities.
A few weeks ago researchers released initial results from a major survey of contemporary American sexual practices, the 2006-2008 National Survey of Family Growth. This study conducted in-person interviews of nearly 14000 prime age men and women and yields what is believed to be the most accurate picture of sexual behavior and sexual history for the current US adult population.
One reason why these results are believed to be especially accurate is that all subjects completed sensitive portions of the interview through audio assisted computer self-interview, or ACASI. What exactly is ACASI? As the authors write, “In ACASI, the respondent listens to the questions through headphones, reads them on the screen, or both, and enters the response directly into the computer. This method avoids asking the respondent to give his or her answers to the interviewer.”
There are several perceived benefits from ACASI. These include:
1. This method may make it easier for persons with lower literacy to complete the self-interview by listening to the questions rather than reading them.
2. Extensive question skip patterns or branching contingent on response that may be confusing in a paper questionnaire (either self-administered or through face to face interview) are handled with ease by the computer.
3. Questions are automatically standardized – every respondent hears the same question asked in the exact same manner.
4. The survey can be administered in a multi-lingual setting with ease.
5. Most germane to the topic at hand, the fact that the response no longer exists on paper with other identifying information, and/or does not need to be recorded by an enumerator, may increase the respondents’ faith in the anonymity of the exercise and result in more accurate reporting.
A study in the late 1990s was perhaps to first to rigorously test ACASI versus more traditional survey methods. In this study, the 1995 National U.S. Survey of Adolescent Males, respondents were randomly assigned to ACASI or to self-administered paper questionnaires.
What did the investigators find? Well, certain non-stigmatized behaviors such as male-female sexual contact were virtually the same with either ACASI or the paper questionnaire. However ACASI respondents were significantly more like to report male-male sexual contact, illicit drug use, and violent behavior. In addition, question non-response was also significantly less – only 0.5% of respondents did not answer a particular question on average versus 2.3% of those with the paper questionnaire.
Of course with this type of inquiry it’s very difficult to validate any reported results, and so higher rates of reported sensitive behaviors are generally believed to be the more accurate given the general downward response bias when anonymity is in question. For example the authors note that the rate of male-male sexual contact measured through ACASI (5.5%) is consistent with other estimates derived from adult reports of adolescent behaviors. In contrast the paper questionnaire estimates the same rate to be 1.5%
So the promise of ACASI in a setting like the U.S. is apparent. But what about in a developing country? I’m sure readers can think of many challenges that an ACASI study may confront. For starters, technological familiarity is presumably a prerequisite for ACASI, and this familiarity is far from universal. One study of ACASI in urban India found a very similar differential performance as found in the US when applied to Indian college students. However male slum dwellers had virtually the same rates of sensitive behavior reporting with either ACASI or face-to-face interview.
More generally, a recent review paper collected results from 15 studies around the developing world that focused on self-reported sexual behavior. Most of these studies contrasted face to face interview with ACASI. By and large the findings are highly inconclusive. Sometimes ACASI resulted in higher reported rates of sensitive behavior and sometimes it did not (the one fairly consistent result is that reports of any forced sex were higher in ACASI than face to face interview).
As might be expected given the technological barriers, ACASI appears to elicit greater divergences in reported rates when applied to urban respondents and those with at least a secondary education. This suggests that the day when ACASI is universally applicable may be far off.
However the world is changing rapidly. Even the most recent study in the review mentioned above is at least 3 years old. Mobile phones are more ubiquitous than ever, even in relatively remote villages. Personal digital assistants are an increasingly viable option for field interview. The technological and behavioral barriers to ACASI are falling. Will we see more widespread adoption of this technology?
All of the examples I cite relate to key public health issues. Presumably there may be potential gains from ACASI in studies of other sensitive topics such as corruption or intra-household relations. If you know of any further work along these lines, please share. We are interested to learn more, as perhaps are many of our readers.
Join the Conversation