One area in which we see very little impact evaluation is the realm of trade related interventions and reforms. In a recent paper Olivier Cadot and coauthors give us a discussion of these types of interventions and how we might evaluate them (they also have an attendant book with some applications).
The need for more evidence is key. As Cadot & co. point out, trade is receiving an increasing amount of policy attention and donors (the World Bank among them) are stepping up support of trade related interventions. But, alas, little work is being done. As a striking example, Cadot & co. review all World Bank trade projects from 1995-2005. Of these 85 projects, only 5 included an impact evaluation that used a comparison group.
Now, the old school trade interventions would make it next to impossible to do a credible impact evaluation. However, these are lessening in importance. As Cadot and co. argue, tariffs are down (for now) with the simple average applied tariff down to about 10 percent from 30 percent in 1990 among developing countries. Quantitative restrictions are also apparently on the decline as the WTO provisions begin to bite.
Instead, trade interventions are becoming more targeted. They focus on two main types of interventions. The first area is trade facilitation which includes customs reforms, and rehabilitation/reforms/construction of ports, airports, railways, and roads. The second area is trade competitiveness which includes interventions such as export promotion, support to producers/exporters, quality certification, credit guarantees and changes to the policy framework. These they argue (rightly) are much more amenable to evaluation.
The paper provides a nice overall guide to methods for the uninitiated and shows how these can be applied to trade policies (with a nice review of some of the papers out there that take a stab at this). For the initiated, there are a couple of points they provide worth thinking about, particularly since some of the goals of these interventions (e.g. those that improve the connection between buyers and sellers) can be found in other areas of policy in which we also see less evaluation.
First, they distinguish between targeted interventions and non-targeted interventions. Targeted interventions such as matching grants or export credits could be amenable to randomized evaluation designs, if this is done as part of ex-ante planning. However, for the broader, non-targeted interventions usually of the trade facilitation flavor (e.g. customs reforms, port upgrading), an RCT approach is usually going to be infeasible – no matter when in the intervention the evaluation enters the picture. Sometimes components can be randomly phased in, but for the most part evaluations of this type of interventions are going to have to rely on other methods.
Second, general equilibrium effects and spillovers are going to be more of an issue in trade interventions than in other types of interventions. As Cadot & co. point out, reforms introduced at say, only one or two border posts (to potentially build some kind of comparison) might result in a shift in trade flows either to, or away from, the treated border posts. Another example is the case of export promotion, where the firms that get the program could possibly hire away skilled workers from other firms and/or demonstrate to comparison group firms the benefits of exporting.
Third, one boon for these type of evaluations is that this is an area where there is an above average level of administrative data (except for those interventions aimed at smuggling of course). So this can be leveraged to reduce the need to collect some types of data. They point out that that some other colleagues at the World Bank are developing a longitudinal exporter level customs transactions database (you can find a presentation using these data here).
Fourth, there are likely to be power issues with these types of evaluation (again, perhaps pushing the evaluator towards firm level matching or some type of regression discontinuity design). For example, if a country is reforming how customs works (e.g. training customs officers, putting in place incentives for non-predatory behavior, and setting up complaint mechanisms) at its border crossings, you might end up with only two main crossing points with one of its major (bordering) partners. Or you could face a small number of firms with export capacity. All of these are going to require some creativity in design, and attention to arguments such as those made by David about increasing the frequency of observations.
So that’s some food for thought. Here’s hoping we see more work on these interventions in the near future. Any examples of good papers on this, or projects in the works are welcome.
Join the Conversation