Devil in the details: measuring agricultural input quality

With guest bloggers: Naureen Karachiwalla (IFPRI), Travis J. Lybbert (University of California at Davis), Hope Michelson (University of Illinois at Urbana-Champaign), Joaquin Sanabria (IFDC), James Stevenson (CGIAR SPIA), and Emilia Tjernstrom (University of Sydney).

In development economics, the issue of measurement can conjure images of either a half-empty or a half-full glass. A stream of recent papers have bolstered skepticism (glass half-empty), by uncovering flaws in how we measure key variables—from self-reported yields to statistics on food consumption and labor supply (blog, paper). On the half-full side, technological innovations continue to expand the frontiers of what and how we measure; we may soon be able to use satellite yields to evaluate crop yields and estimate rural poverty in low-income countries.

Laboratory-based measurement have brought these tools within reach of economist research budgets, including our own. These tools can enable more accurate assessment of the quality of agricultural inputs (including fertilizers, herbicides, and maize seeds), to inform our understanding of the economics of actual and perceived quality on farmer choices. This goes beyond the academic sphere. For Joaquin, these methods have been central to his work measuring fertilizer quality across sub-Saharan Africa (SSA) at the International Fertilizer Development Center (IFDC).

By revealed preference, we’re enthusiastic about the possibilities for these tools to reduce uncertainty. However, experiences in collecting, analyzing, and interpreting this class of data compels us to sound a cautionary note. We start with urea fertilizer. A follow-up post will discuss seeds, which raise even more complicated quality and measurement issues.

Measure twice, interpret once?

There are many kinds and formulations of fertilizer. We focus here on synthetic urea, the most common variety used and sold in Sub-Saharan Africa. Urea should contain 46% nitrogen by weight. If they exist, systemic deficiencies in urea’s nitrogen content could have consequences for regional agricultural productivity and food security.

Farmers often express distrust in the quality of fertilizer available in local markets; indeed, recent research quantifies the level of distrust in Uganda, and two studies in Tanzania. But do farmers’ beliefs correspond with reality and, if not, what are the resulting welfare implications?

In 2016 Hope and co-authors collected and tested 421 samples of urea directly from agro-dealers in rural Tanzania. The team shipped samples for analysis to a certified East African laboratory, and—following best-practices—also sent a random 10% of these samples for double-testing at a US-based lab. The results from the East African laboratory suggested considerable nitrogen deficiencies, with some samples evidently containing less than 30% nitrogen (remember, the concentration is supposed to be 46%) – consistent with farmer distrust. In stark contrast, the US-based lab results showed no evidence of nutrient content shortfalls in any of the samples. Moreover, when Hope’s research team worked with the East African lab to assure instrument calibration and reduce other error sources, a second round of testing (figure below) found nearly all of the samples to be within compliance standards (i.e., at least 45% nitrogen).

With urea testing in Uganda, Naureen and co-authors had a nearly identical experience. A first round of testing suggested big nitrogen shortfalls but double-testing indicated that the urea was excellent with no nitrogen problems. As it turns out, most of the available evidence finds that urea is consistently high quality (including IFDC’s systematic quality assessments in SSA, and a study by Emilia and Travis that tested 600 samples of fertilizer in western Kenya). The IFDC notes that out-of-compliance urea samples are nearly always the result of problems with the chemical analysis.

A single study conducted in Uganda by Bold and co-authors finds dramatic deviations: an average shortfall of 30% of nitrogen across 369 tested urea samples. However, this study reports testing of these samples in a single lab; moreover, the study defines a samples’ quality as an average of three measurements but does not document the variance of these multiple measurements, nor does it report the overall analytical error. The findings are an outlier in a literature that finds no problems with urea across countries and time. Two other studies conducted in Uganda by Joaquin’s IFDC team and Naureen and co-authors, with similar timing and sample-acquisition strategies, find no nutrient deficiency problems in locally-marketed urea fertilizer in Uganda.

This might seem counterintuitive: in a context where low-quality products are commonplace, why would urea be an exception?  Two answers: first, intentional dilution of urea is unlikely profitable. Would-be swindlers have two options: they can either re-granulate or mix in cheaper additives; however, re-granulation is technologically challenging and energy-intensive and few additives would reliably fool purchasers, since urea fertilizer consists of evenly-colored, crystalline spheres. Second, degradation is less likely than one might think: urea is a very stable molecule unless it’s combined with the urease enzyme, which is found in soil and plants—not in the air of agro-dealers or warehouses.

While urea is among the least expensive fertilizers in the market, it may be more profitable to dilute higher-value, multi-nutrient fertilizers. These variants could be more susceptible to manufacturing deficiencies.

The economics of perceptions about fertilizer quality

There are potentially major implications for fertilizer use from a disconnect between actual and perceptions of fertilizer quality. Despite finding that urea fertilizer in rural markets generally meets standards, farmers remain suspicious and this in turn depresses fertilizer demand. This finding opens up questions related to farmer beliefs: where do they come from and why do they persist? Farmers operate in markets characterized by asymmetric information and weak, under-resourced regulatory systems. Moreover, work by Hope and co-authors shows that much of the fertilizer available in rural markets looks bad, as it is often clumped or dirty, sometimes with bits of maize or grass in the mix.  Though it’s well-established that such observable degradation doesn’t affect the urea molecule or the nitrogen content, farmers can be skeptical about what they see in an open bag. New work by Hope and co-authors shows that such incorrect and economically harmful beliefs about poor quality can persist in a context in which misattribution (i.e., farmers attributing a bad yield to poor-quality fertilizer rather than other growing conditions) and uncertainty about beliefs prevent learning.

As with fertilizer, perception of seed quality influences uptake but seed quality is more complicated to measure. Stay tuned for a follow-up blog on this area, with reflection on what these new opportunities in measurement may mean for development economics research.