Surveys around the globe are used to measure specific outcomes, relying on carefully crafted questions to capture very precise concepts. However, survey respondents’ understanding of these questions, and the variation in that understanding, is often unknown. In this context, cognitive interviewing can be a powerful tool to assess how well and how consistently respondents are understanding survey questions and concepts, ultimately allowing practitioners improve questionnaire design. The recently published Companion Guide for Cognitive Interviewing guides survey practitioners in implementing cognitive interviewing specifically related to individual land tenure rights, based on experience in Armenia.
This companion guide facilitates cognitive interviewing in preparation for the implementation of the questionnaire module on measuring land tenure at the individual level for monitoring SDGs 1.4.2 and 5.a.1 (FAO, World Bank, and UN-Habitat, 2019). By implementing a cognitive interviewing phase prior to the implementation of the land tenure module, practitioners can, in a cost-effective manner, ensure the module is properly designed, translated, and implemented for the specific context. However, revisions to this particular questionnaire module should be aimed at minor adaptations for the context and translation, as the questionnaire has been carefully designed to meet the needs of the SDGs.
Cognitive Interviewing 101
Cognitive interviewing typically entails a small number of intensive, focused in-depth interviews and discussions aimed at uncovering respondents’ cognitive processes associated with specific survey questions. It is implemented in advance of survey fieldwork to allow time for questionnaire modifications, as needed. In the companion guide for cognitive interviewing for the land tenure module, as seen in Figure 1, we recommend the implementation of: (i) individual in-depth interviews, which includes administration of the land tenure module questions themselves plus predetermined probing questions aimed at determining how understandable the questions were, what the respondent’s attitude was towards a particular question, the potential difficulties in answering the questions, etc.; and (ii) focus group discussions, which serve as a useful supplement to the in-depth interviews as they allow a group of respondents to talk through their interpretations of the questions, highlighting potential differences in understanding.
Why Cognitive Interviewing around Land Tenure?
The concepts underlying land tenure are complex. Couple those complexities with the nuances of the SDG 1.4.2 and 5.a.1 indicator definitions and the various methods for data collection, and you have numerous possibilities for the data collected to differ from the data you intended to collect. For example, because the intention is to understand individual land rights, as opposed to household-level land rights, it is imperative that survey respondents interpret the questions such that they answer only about their own personal rights. Due to cultural norms, it may be that in some contexts men and women interpret and respond to these questions differently.
Additionally, the questionnaire module on land tenure includes options for interviewing a self-respondent (i.e., respondent answers only on behalf of themselves), the recommended approach where possible, or a proxy respondent (i.e., respondent answers on behalf of themselves as well as other household members), as well as options for collecting data at the parcel level or an aggregated level. The companion guide offers support for cognitive interviewing around each of these options, to enable practitioners to understand how the choice of these survey designs may impact data quality in a given context.
In the companion guide you will find…
- An overview of cognitive interviewing
- Indications of staffing and sampling requirements
- Design and findings of the cognitive interviewing component of the Armenia Land Tenure and Area (ALTA) study
- Cognitive interviewing templates specifically targeted at the land tenure module for measuring SDGs 1.4.2 and 5.a.1
Resources:
Read the guide to cognitive interviewing here.
Find the questionnaire module for measuring SDGs 1.4.2 and 5.a.1 here
For more information on the ALTA study, visit https://www.worldbank.org/en/programs/lsms/land-tenure
Join the Conversation