Quy-Toan Do (World Bank), with Andrei Levchenko (University of Michigan) and Lin Ma (National University of Singapore)
As the Convention on International Trade in Endangered Species of Wild Fauna and Flora (CITES) convenes its 17th Conference of the Parties later this month, the elephant conservation policy space continues to be polarized, with some countries advocating for a continuation of the complete ban on international legal trade in ivory while others, such as Namibia and Zimbabwe proposing to resume a regulated international trade in their legal ivory stocks. The legal ivory trade is generally opposed by countries with small or declining elephant populations that are against the consumptive use of wildlife. They fear that a legal trade will increase demand for ivory and thereby increase poaching in their countries. On the other hand, the legal trade is supported by countries with stable or growing elephant populations, who believe in sustainable consumptive use. They feel that a continued ban on the ivory trade penalizes them for their conservation successes and removes an important incentive for the conservation of elephants and other wildlife and their habitats by providing funding for management and incentives to local communities.
Conceptually, it is not clear whether legal ivory sales would help or hurt elephant populations. At the most basic level, legal sales should crowd out demand and therefore reduce prices and poaching. On the other hand, some argued that a legal trade increases the possibility that poached ivory would be laundered through the newly legal channels or that it would legitimize consumption in the eyes of customers, hence inducing an expansion of the demand. The impact of legalizing the ivory market on the survival of the species is thus theoretically ambiguous, and is therefore an empirical question.
A recent working paper adds much-needed evidence to the debate. The authors, Solomon Hsiang and Nitin Sekar, use the 2008 one-time legal sale of ivory as an experiment, and evaluate its impact on poaching. The authors’ main finding is that the increase in legal supply was accompanied by a strong increase in elephant poaching. The authors’ preferred interpretation for this result is that a legal sale raises demand by de-stigmatizing ivory consumption and by making poached ivory smuggling easier. Their findings have clear and far-reaching policy implications, and were received with great interest by policymakers and the media (see e.g. the Guardian) and --- as is precisely what discussion papers are for -- intense discussion on the scientific merit of the paper ensued.
One blogpost in particular caught our attention: Fiona Underwood replicates the results in Hsiang and Sekar (2016) and finds that these are sensitive to their estimation strategy, in particular whether or not sampling weights are applied. To try to better understand the Hsiang-Sekar finding and Underwood’s rebuttal, we downloaded the MIKE illegal poaching data used in both analyses. The dataset is a product of the Monitoring the Illegal Killing of Elephants programme of CITES and aggregates data collected by anti-poaching patrols and other sources. The PIKE variable, the proportion of illegally killed elephants, is constructed as the ratio between the number of carcasses illegally killed and the total number of carcasses encountered. It is important to note that the poaching sites that comprise the MIKE database are very heterogeneous. The total number of carcasses (the denominator in the PIKE ratio variable) ranges from 0 to 310. Out of the total 77 sites available in the MIKE data, 24 have an average of less than 3 carcasses. At the same time, 7 sites have an average of more than 50 carcasses. These few sites account for 51.5% of all identified illegally-killed elephant carcasses in the dataset. In other words, while there are many sites in the MIKE database, the bulk of the total elephant poaching occurs at the large sites.
Hsiang and Sekar’s finding is that the average PIKE experienced a step increase in 2008, the year coinciding with the legal sale. Underwood however finds that once the data are weighted by the total number of carcasses, Hsiang and Sekar’s results no longer hold. The question of whether to weight data in regression analysis is not new. Gary Solon, Steven Haider, and Jeffrey Wooldridge (2015) discuss the rationales for using sampling weights in empirical analyses and Jed Friedman reflected on that paper a few years back on this very blog. In this instance, if we wanted to know how much total poaching increased in 2008, the natural sampling weight would be the population size of elephants at a given site divided by the number of carcasses detected. While the total number of carcasses can be expected to be correlated with elephant population, it also depends on accessibility (forest vs. savannah), resources for patrols, etc.; the data are thus not self-weighted. The question of the appropriate weighting strategy, therefore, is not straightforward. What we know nevertheless is that weighting is irrelevant if the treatment effect is homogeneous across sites and over time.
This turns out not to be the case: our analysis indicates that the increase in PIKE in 2008 is confined to the smallest sites. The four panels in Figure 1 below show the average PIKE in each year after having filtered out site-level characteristics, as both authors did. Each point is the average, while the bracket indicates the one standard deviation band. Panel (a) replicates Hsiang and Sekar’s analysis where a jump in PIKE is apparent in 2008, while panel (b) replicates Underwood’s in that observations from a given site are weighted by the total number of carcasses found at the site. As reported earlier, such weighting strategy leads to no discernible difference between the pre- and post-2008 periods beyond a steady upward trend.
When we split the sample into small and large sites, the result that PIKE increases is only present in the subsample of small sites, irrespective of whether we classify a site as small versus large using a cutoff of 2 carcasses (1/3 of the data), or 20 carcasses (2/3 of the data), or anything in-between. Panels (c) and (d) show, for the 20-carcass cutoff, that the patterns pictured in Panel (a) are indeed driven by small sites only. In large poaching sites, the Hsiang-Sekar result does not hold.
What do these results suggest? In the presence of heterogeneity across sites, a simple unweighted average across sites is misleading. Because large sites are where most of the overall poaching takes place, the figures in Panels (b) or (d) above are thus better description of what happened on the aggregate than Panels (a) or (c) are: basically nothing happened beyond the ongoing upward trend.
We can nonetheless elaborate on what we believe might or might not have happened around 2008 at small sites. Admittedly, the impact described by Hsiang and Sekar could have been felt in small sites only. In other words, the supply of poached ivory could be more price-elastic in small than it is in large sites. This would be the case if poaching is easier in smaller sites due to e.g. lower law enforcement presence. Nonetheless, a necessary condition for poaching to increase is that poachers faced increased prices.
In 2015, the World Bank commissioned TRAFFIC to collect ivory price data from the Elephant Trade Information System’s (ETIS) seizure records, published reports and articles, contemporary websites, customs statistics from various countries, some proprietary industry- or government-held datasets, and unpublished TRAFFIC market monitoring survey data. Members of the IUCN/SSC African Elephant Specialist Group were also asked for data and a few data points were obtained accordingly. As a result, the final product is a dataset containing 21,395 ivory price observations from 86 countries and over the period 1970-2015, making such dataset the largest collection of ivory price data in the world. Looking solely at the 4,873 records on raw ivory, we constructed Figure 2 below, which plots the time series of the (log) median unworked ivory prices in sub-Saharan Africa, and in China (inclusive of Hong Kong SAR, China), which was the largest market for ivory over this period. While there are clear trends in the ivory prices over time, it is notable that neither price series shows a marked change in 2008, casting some doubt on the claim that the 2008 sale had had any impact on illegal ivory markets.
An alternative explanation is that Hsiang and Sekar’s findings for small sites are a statistical artefact. PIKE is the ratio of the number of carcasses from illegally-killed elephants to the total number of carcasses. The construction of this measure requires expertise of MIKE monitors on the ground who receive training and exercise judgment to determine whether a newly discovered carcass comes from an illegally-killed elephant or not. Carcasses therefore can be misclassified one way or the other, and small changes in classifications can have large effects on PIKE, especially in smallest sites. For example, a site with a total of 3 carcasses with one classified as “illegal kill” will have a PIKE value of 1/3. A reclassification of one carcass only would bring PIKE to 2/3: a 100% increase in measured poaching! Such sensitivity eventually vanishes for largest sites. Figure 3 below replicates Hsiang and Sekar’s analysis by randomly re-classifying carcasses from illegal to legal in small sites (two total carcasses or less) in 2008. Each point is based on 200 random re-classifications. The figure shows that the reclassification of only 7 carcasses from illegal to legal death is all that is needed for the 2008 step increase to no longer be statistically significant at conventional levels (10 percent as per Hsiang and Sekar’s analysis). Seven carcasses is a small fraction of the nearly 1,000 carcasses found in MIKE data for 2008, among which around 500 are classified as illegally killed.
We have reasons to believe that the classification of carcasses might have experienced some changes around 2008. The MIKE programme received a large grant through an ACP project from the European Development Fund. The training of and additional resources for rangers that could be funded might have led to increased classification of carcasses as illegal kills; in particular, CITES reports a gradual decline over time in the proportion of carcasses for which the cause of death is either recorded as “unknown” or is missing, which, for the construction of PIKE, is labeled as “not illegally killed” (as no evidence of illegal killing was reported).
In this short discussion, we argued that the results reported by Hsiang and Sekar are confined to only the smallest sites surveyed by the MIKE programme. Aggregate poaching therefore did not experience a step increase in 2008 as argued by the authors. Our data on raw ivory prices in both Africa and China further support the conclusion that the 2008 one-off sale actually had no discernible effects on ivory markets. Rather, we postulate that small changes in the classification of carcasses could account for the results documented by Hsiang and Sekar.
UPDATE: Hsiang and Sekar respond, and Do and co-authors respond back.
Join the Conversation