Agricultural Survey Design: Lessons from the LSMS-ISA and Beyond

Household surveys are the primary tool through which international development goals are monitored, and policy questions are informed based on empirical research. While the importance of agricultural statistics is widely acknowledged, practices in data collection vary widely across national statistical offices, international organizations that generate international data public goods, NGOs, and researchers.

The World Bank’s Living Standards Measurement Study (LSMS) team developed a guidebook on Agricultural Survey Design, a consolidated reference for research­ers, survey practitioners, and policy analysts on the design of agricultural surveys, taking into consideration the wealth of knowledge gleaned from methodological research and innovation in the two decades since the seminal book on Designing Household Survey Questionnaires for Developing Countries by Grosh and Glewwe (2000). Informed by the survey design literature and the experience of the LSMS-Integrated Surveys on Agriculture, this guidebook presents best practices for multiple components of agricultural survey design, including units of analysis, respondent selection, and recall periods, as well as detailed discussions on numerous agricultural topics. Reference questionnaires for crop and livestock agriculture are provided in the guidebook, with detailed discussions in each chapter linking the literature and the questionnaires. The guidebook also highlights topics for which more research is needed to inform best practices.

Agricultural Survey Design guides users in answering critical questions for collecting high-quality agricultural data, such as:

• What is the appropriate recall period for collecting data on crop agriculture? Recall periods for agricultural statistics require consid­eration of the different frequencies and seasonal timing of input decisions and harvest periods. Survey visits may be employed once per year, in two visits per agricultural season, or in more than two visits. The more frequent the survey visits, the less recall the survey requires. The implementation of a two-visit approach where feasible is highly recommended, as this offers a balance of practical implementation feasibility and limited recall bias. The reference questionnaire provided in Agricultural Survey Design is based on a two-visit approach, collecting data following planting and harvesting.

• How do I collect the most accurate data on crop yields? Accurately measuring yields, a principal agricultural statistic, requires sound data on crop production and land area. Agricultural Survey Design discusses the benefits of collecting data at the parcel-plot-crop level, to better link production and inputs, including land area, as well as allowing for the analysis of crop yields by plot manager characteristics, such as gender. The methodological options for measuring crop production at the parcel-plot-crop level, including through respondent estimate, remote sensing, and crop-cutting are presented in the guidebook, as well as the tradeoffs of each approach. Though surveys often rely on respondent estimates of land area, objective methods of land area measurement, primarily through the use of handheld GPS devices, is recommended in the guidebook, as supported by a growing body of methodological research.

• What challenges should I expect in collecting data on livestock? The nature of livestock activities, particularly among smallholders, where many rural households maintain herds away from the household and livestock sales may be infrequent, renders data collection difficult. Mortality and sales may be under-reported due to recall challenges and/or stigma associated with animal mortality. Additionally, many livestock sector outcomes should be recorded at the herd level rather than the household level, particularly for nomadic populations. Household surveys with a population-based sampling will never capture herd level outcomes for which area-based sampling measures may be a better unit of analysis. The livestock questionnaire provided in Agricultural Survey Design, drawn from the LSMS guidebook on Measuring  the Role of Livestock in the Household Economy, assists readers with collecting livestock data according to best practices.

• What are the practical considerations for implementing agricultural surveys? Agricultural Survey Design details the essential considerations when planning and implementing agricultural surveys, including the value of piloting, training needs, number and timing of survey visits, and fieldwork team structure. Additionally, the trade-offs associated with various data collection modes are discussed, focused primarily on paper-assisted personal interviewing (PAPI) and computer-assisted personal interviewing (CAPI).

See the full Agricultural Survey Design guidebook for detailed discussions and consolidated guidance on (i) survey design choices for agricultural surveys; (ii) the measurement of crop production, agricultural inputs, and livestock; and (iii) practical considerations for the implementation of agricultural surveys.

Resources:

Read the full guidebook here