There are a fair number of interventions out there that work with an entire value chain with a set of interventions. The first (and second) time I was asked to evaluate one of these, my response was how hard, even impossible, it might be. I have since been enlightened, first with David’s post on Monday and also from reading an exciting new paper by Macchiavello and Miquel-Florensa.
Macchiavello and Miquel-Florensa evaluate a commercial program in Colombia. The goal of the program is to get more sustainably sourced, higher quality coffee. As such it works straight through the whole value chain – exporters, mills, cooperatives and farmers. There isn’t an obvious control group. But Macchiavello and Miquel-Florensa take a super creative approach and pull out rigorous and interesting answers. And, as a bonus, they do this with mostly secondary or administrative data.
Let’s start with the program. It’s run by the mysterious international “Buyer” (for confidentiality reasons they don’t reveal the name of the company in the paper). The program (it’s called the sustainable quality program) tackles both the supply and demand side. On the supply side, it provides training, agricultural extension and access to inputs to help renew plots. On the demand side, it offers farmers a fixed price premium for beans that meet quality standards, without a pre-commitment to sell. The program also works with exporters and cooperatives – paying for the supply side interventions and contracting the export prices (among other things – this isn’t fully observed).
The program selects eligible veredas, which are small administrative units which comprise 60-70 plots, based on their terroir. To join the program, a farmer must have plots that meet quality and environmental conditions. This may entail planting new trees and upgrading the field. Macchiavello and Miquel-Florensa use the roll out of the program across space and time to build a difference in difference identification strategy for the initial estimates of impacts on farmers. The first thing they bring to bear are plot fixed effects to deal with underlying land characteristics. They also add municipality-year fixed effects to deal with factors that could be driving upgrading or program roll out at the municipality level. Finally, since there could be self-selection at the farmer level, they do most of their estimates with an ITT approach at the vereda.
In terms of data Macchiavello and Miquel-Florensa are using a range of different sources. First up is a geo-reference panel that covers all coffee plots in Colombia during 2006-2016. This is collected by the national coffee federation (the exporter in this case) and has a number of variables on the condition of the plot, a bit on the farmer but nothing on sales and production. For this Macchiavello and Miquel-Florensa turn to data from one of the participating cooperatives on sales (for 2015-16) and transaction level sales for any farmers and buyers in 2013. And these data have good information on origin and quality too. It’s interesting to note that none of their data comes from the international buyer.
So what do they find? First of all, farmers upgrade the quality of their plots in terms of younger and more disease resistant trees to the tune of 0.047 standard deviations (these are ITT estimates). Second, they increase the area cultivated with coffee by 2 percent. They also bring in new plots altogether with a 9 percent increase in land under coffee. So more coffee on better fields.
And is it good coffee? Indeed it is. Macchiavello and Miquel-Florensa use buying point-season fixed effects, as well as month fixed effects to look at this and find that batches of coffee sourced in program areas have a 0.42 standard deviation higher quality (the index uses things like the cupping test). One might be concerned that this is just a sorting effect – the good coffee in program areas is going to program buyers and that means the other coffee in that area is actually worse. Macchiavello and Miquel-Florensa look at this and find nothing.
Now let’s talk prices. Based on my experience working on pineapple, one problem farmers face is that when it comes time to buy, your exporter may just renege. Macchiavello and Miquel-Florensa look at this and see that 90 percent of the farmers who made any delivery to the cooperative for the program had their coffee purchased. And they got 9.6 percent higher prices – a result that’s robust to farmer, and farmer-season fixed effects – and what the program promised (10 percent). Macchiavello and Miquel-Florensa also look at whether farmers engaged in side-selling. Here they roll out a spatial discontinuity design, comparing farmers on both sides of the boundaries of program areas. And it looks the folks in program areas are selling to the program, not to others.
Macchiavello and Miquel-Florensa then build a model, and calibrate it with their data and results, to look at the impacts throughout the value chain. I won’t go through the details of the model here, but it yields estimates that welfare for the eligible farmers goes up by 19 percent. Macchiavello and Miquel-Florensa estimate that the interventions increased surplus in program areas by 33 percent, with farmers keeping 56 percent of the surplus increase, with the rest going to the exporter. One point they make is that contract structure between the international buyer and the exporter (which specified the premium to be given at the farm gate) helped avoid the case where the exporter could have set an inefficiently low farm gate premium.
But wait, that’s not all. Macchiavello and Miquel-Florensa then try to shed some insight into the mechanisms of how the program worked. Obviously, they are observing it in its entirety, but they take two approaches to take a stab at this. First, they run a bunch of counterfactuals with changes in supply, demand and market structure. But they also compare the prices along the chain for this program with two other voluntary sustainable standards (VSS) programs – one which focuses on environmental sustainability and the other on social. They are both implemented by the same agency (the national coffee federation) that the program Macchiavello and Miquel-Florensa are looking at.
This last exercise gives them two main insights. First, the contract structure that the international buyer put in place matters. One way they show this is that In the other (VSS) schemes very little of the price premium makes it from the exporter level to the farm gate – in contrast to the program here where about 70 percent of it does. So not all fair or eco trade coffee is equal (for the farmer).
Macchiavello and Miquel-Florensa also argue that program is dealing with a market failure. Using price differentials as well as cost calculations, they show pre-existing inefficiency within the market and argue that the program helped alleviate market power and limited contract enforcement distortions in the market.
This is quite a complicated paper, with many interesting twists and turns. And more than one identification strategy. There is a bunch to learn. First, there are neat lessons on program design and, while all contexts are going to be somewhat different, the incentives and the types of market failures are going to be less so. And this gives an example of a program implemented by a really large buyer which can translate into better prices and welfare for farmers. Methodologically, it’s an instructive paper in how creative use of data that is part of this commercial ecosystem can be used to get a handle on an intervention that tackles multiple levels of the market.
Join the Conversation