Spillovers are a familiar challenge for development economists. Measuring these indirect effects on non-targeted individuals, firms, locations, or time periods can be difficult, but doing so is often necessary for uncovering the total impact of a policy or program. One setting where spillovers are particularly inherent and acute is marine (ocean) fisheries. The high mobility of fish and fishing vessels necessitates addressing spillovers in nearly all empirical fisheries research. Below, I discuss three papers with distinct spillover concerns, and how these studies address them.
Directional treatment effects
Sometimes the relevant research question is learning the direction of a treatment effect. In this scenario, we learn whether an intervention increases or decreases the outcome of interest, while the magnitude of the treatment effect remains unknown.
Sarah Medoff, John Lynham, and Jennifer Raynor (2022) investigate the conservation benefits of Marine Protected Areas (MPAs). MPAs limit or prohibit fishing in a location. They are premised on the idea that such spatial restrictions will enable fish populations to increase inside the MPA. In this scenario, the density of mobile fish species should also increase outside MPAs as fish disperse from the MPA. Thus, increases in fish density outside MPAs would provide evidence that MPAs increase fish populations.
To test this hypothesis, the authors compare a proxy for fish density near versus far from a large MPA, before and after MPA implementation. This difference-in-differences design recovers the direction of the effect, as near areas should experience larger increases in fish density than far areas. However, due to the continuous nature of fish movement, the MPA likely also increases fish density in far areas compared to the counterfactual in which no MPA was implemented—our spillover of concern. Therefore, the authors’ estimates are likely a lower bound on the true effect, since the increase in near areas is compared to a smaller increase in far areas, rather than to a counterfactual unaffected by the MPA.
Pure controls unaffected by spillovers
One way to avoid contamination of treatment effects is by choosing a control group that is unaffected by spillovers. Eyal Frank and Kimberly Oremus (2023) implement this approach in their evaluation of a United States (US) law requiring rebuilding of depleted fish populations. After the law’s passage in 1996, the depletion of a US fish population below a predetermined scientific threshold triggered regulators to reduce total allowed catch until the fish population had increased to a biologically healthy size.
Comparing treated US fish populations to contemporaneously untreated US fish populations would not recover the causal effect of the law, since US fishers often catch multiple species and US fish populations interact with each other. For example, the authors identify 20 US fish populations that could have been treated, but were not because scientific thresholds were only recently determined for them. These “placebo” populations exhibit similar size increases as treated fish populations because they are affected by the treatment status of fish populations whose habitats overlap.
Instead, the authors compare treated US fish populations to the same species in the same years in the European Union (EU). The EU enacted a similar law two decades after the US. During this period, the authors choose as their control group EU fish populations that are spatially separated from their US counterparts. These EU fish populations would have been treated in a similar manner had the EU implemented the law sooner. The authors provide convincing evidence against spillovers contaminating their control group, enabling them to demonstrate that the 1996 law increased treated fish population sizes by 50%.
Combining spillover and direct effects to estimate the total treatment effect
The third approach identifies a pure control group, but seeks to add the spillover effects on non-targeted units to the direct effect on targeted units. The underlying rationale is that spillovers are an integral part of the intervention’s effects and should be accounted for when measuring the total treatment effect.
Temporary spatial closures can be thought of as MPAs with flexible boundaries that turn on and off in response to information received by the regulator. In the Peruvian anchoveta fishery, the regulator aims to reduce the capture of juvenile (sexually immature) anchoveta by temporarily prohibiting fishing in areas where the share of juvenile catch is high. Due to the mobility of fish and fishing vessels, it is essential to consider not only what happens inside temporary spatial closures during the closure period but also the effects outside and after the closures.
My 2023 paper finds that while juvenile catch decreases inside closed areas during the closure period, it increases outside them during the closure period as well as inside them after the closure period. These spatial and temporal spillovers more than offset the direct decrease in juvenile catch, resulting in a positive total treatment effect (an unintended increase in juvenile catch). If the estimation only compared the change in juvenile catch inside directly targeted units to similar control areas, it would have incorrectly suggested that the policy reduces juvenile catch. By forming the total treatment effect as the sum of direct and spillover effects, I conclude that the policy’s total impact is the opposite of its intended goal.
Embracing spillovers in the research design
Spillovers are an inherent challenge in many development economics settings, but addressing them is often crucial for accurately measuring the total impact of policies or programs. When spillovers are likely occurring, development economists can and do directly incorporate them into their research design and interpretation of treatment effects. The highly mobile and interconnected nature of marine fisheries demonstrates that obtaining interpretable treatment effects is possible even in complex environments. By embracing spillovers as an integral part of many interventions’ effects, researchers can measure total causal effects, ultimately providing policymakers with more accurate and comprehensive information to guide decision-making.
Join the Conversation