Can satellites deliver accurate measures of crop yields in smallholder farming systems?

How much food is produced on a plot of land? The answer is central to several pressing questions in agricultural and development economics: How efficiently do smallholders use their labor and land? What interventions are most effective at lifting smallholders out of poverty? Are smallholders better off investing more time and resources on the farm, or intensifying their reliance on off-farm employment? The answers in part depend on the ability to accurately measure crop production. This is why household and farm surveys across the developing world, such as those supported by the World Bank Living Standards Measurement Study – Integrated Surveys on Agriculture (LSMS-ISA) initiative, attempt to obtain precise, within-farm measures of crop production and productivity.

In the context of the LSMS-ISA, apart from improvements in land area measurement (based on the research summarized here), production information is still based on farmer recall (except for Ethiopia and Mali, relying on sub-plot crop cutting). Recent research (ours in Uganda; by Desiere and Jolliffe for Ethiopia; and by Abay et al. for Ethiopia) showcased the non-random, systematic errors in farmer-reported production information. This is of course against the backdrop of well-documented, persistent weaknesses in agricultural data in low- and middle-income countries.
 
A possible alternative is to try estimating crop yields from satellite imagery. There’s a long history of using satellites to assess crop condition or yield, with considerable recent progress. However, this work has mostly focused on large commercial farms or on estimating aggregate yields over large regions. Getting plot-level precision for smallholder farmers has been beyond reach, mainly because the resolution on widely used sensors was too coarse to distinguish individual fields.
 
The good news is that things are quickly changing, with several new satellite sensors launched in recent years. In our latest paper, our group of Stanford and World Bank researchers tried to see just how well these new satellites can work for estimating maize yield, relative to ground-based measures that rely on farmer reporting; sub-plot crop cutting; and full-plot crop cutting.
 
Our analysis is using the data from the 2016 round of MAPS: Methodological Experiment on Measuring Maize Productivity, Soil Fertility, which was conducted by the Uganda Bureau of Statistics (UBOS) in Iganga and Mayuge districts of Eastern Uganda. Our paper builds on the earlier work by Burke and Lobell for Kenya, and we summarize the main takeaways below.
 
First, to give a sense of how much better the new satellite data are, the figure below shows a small part of our study region as viewed by (1) Landsat (the old mainstay of land remote sensing, taking imagery every 16 days); (2) Sentinel-2 (a pair of European sensors launched in 2015-16, taking imagery every 5 days); and (3) Skysat (a set of sensors owned by Planet, a private firm in San Francisco that graciously collected data for our study region during the 2016 growing season). You can see how individual fields become much clearer at 10m than 30m, whereas the 1m data provide fine detail on both fields and individual buildings.



While the paper provides the full details regarding the (panel) sampling design; the questionnaire instruments; and the survey methods, the key features of our study were as follows. 

• 463 plots were visited, and the field boundaries were collected with a GPS.
• On each plot, an 8x8 m sub-plot was set-up at random during the post-planting period, and was later harvested, dried, and weighed, allowing for the computation of the sub-plot crop cut (CC) yield.
• Approximately half of the plot sample (211 plots) was selected at random for a full plot harvest. The weight of the harvest, combined with the GPS-based plot area, provided the full-plot crop cut (FP) yield.
• Self-reported maize yields were obtained for the remaining 252 plots by dividing farmer-reported production (converted to grain, KG-equivalent terms) with GPS-based plot area. 

This design represented a heroic amount of effort in the field, and it was masterfully executed by highly-trained UBOS field teams. Often the hardest part in telling whether satellite yields are accurate is having something good from the ground to compare it to. It is extremely rare to have such rich datasets of both subjective and objective yield measures.
 
Fortunately, we were able to obtain three relatively cloud-free Sentinel-2 images during the season. In the tropics, clear skies can be hard to find, even at the early satellite overpass time of 10:30 am, so even the best images can have clouds as seen below (the plot locations in yellow).



Using these images, we applied methods developed in our group to estimate maize yields, and then compared them with the ground-based measures of yields and inputs. Satellite yields include two versions that were calibrated to FP and CC yields, and an alternative based on crop model simulations, using no ground data (uncalibrated), following Lobell et al. (2015).
 
The headline findings of our analysis were: 

• Self-reported yields appeared very unreliable in this region – explaining less than 1 percent of crop cutting-based yield variance.
• CC and FP yields correlated well with each other across plots, but not as strong as one might wish for (r = 0.5). This signals substantial sub-plot yield variability even on small plots, and how difficult it still is to get accurate yield measures using sub-plot crop cutting.
• Both calibrated and uncalibrated satellite yields capture roughly half of the variance in FP yields on pure stand plots > 0.1 hectare – a very promising finding that one could have only imagined just 5 years ago.
• Compared to CC and FP yields, the use of satellite yields in yield analysis faithfully reproduced the effects of production factors such as (objective) soil quality and fertilizer use.
• Although CC yields are imperfect approximations of plot-level yields, the errors do not substantially bias remote sensing calibrations. Sub-plot crop cutting, therefore, appears to be a suitable replacement for full-plot harvests.

Even though we placed emphasis on measuring yields at the plot-level, we recognize that many applications will care more about accuracies at aggregate scales. What is expected to become increasingly more useful will be the ability to integrate georeferenced micro survey data on agriculture, such as the LSMS-ISA, with the expanding, publicly-available high-resolution satellite imagery. Such ability, combined with advances in remote sensing methods, has the potential to create an unparalleled scope for research on entire landscapes of agricultural plots.
 
As always, there are lots more questions to answer. But the answer to the title of this post seems to be “yes.” Please check out the paper for more details.