Worm Wars: The Anthology
This page in:
(This post was updated on January 4, 2016.)
Over the last several weeks, tens of thousands of words have been published about a study on the benefits of deworming for Kenyan schoolchildren, about the benefits of deworming more generally, about replication in science and social science, and about the evidence base for development programs. More will surely be written. As you’ll see below, several blog posts seek to make sense of the hubbub.
For those engaged or interested in the debate, I provide links to the primary research documents as well as many of the responses.
The key research documents for the current round of the Worm Wars (there were earlier versions of the replication and responses to those, which I list below)
- The original paper – Miguel & Kremer’s Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities, Econometrica (2004)
- Aiken et al. – Re-analysis of health and educational impacts of a school-based deworming programme in western Kenya: a pure replication, International Journal of Epidemiology (2015)
- Davey et al. – Re-analysis of health and educational impacts of a school-based deworming programme in western Kenya: a statistical replication of a cluster quasi-randomized stepped-wedge trial, International Journal of Epidemiology (2015)
- Response to Aiken et al. & Davey et al. by Hamory Hicks, Kremer, and Miguel, International Journal of Epidemiology (2015)
- Hargreaves et al. (replication authors' response to Hamory Hicks, Kremer, and Miguel) -- Authors’ Response to: Deworming externalities and school impacts in Kenya, International Journal of Epidemiology (2015)
- Newest Cochrane review – Taylor-Robinson et al. – Deworming school children in developing countries (2015)
- Goldacre – Scientists Are Hoarding Data And It’s Ruining Medical Research – BuzzFeed, 2015
- Boseley – New research debunks merits of global deworming programmes – The Guardian, 2015
- Blattman (Columbia U) – Dear journalists and policymakers: What you need to know about the Worm Wars (7/23/2015)
- Evidence Action blog – Worms Win, Kids Lose? Our Statement (7/23/2015)
- Wood (3ie) -- Replication research promotes open discourse (7/23/2015)
- Kremer & Miguel (Harvard / Berkeley) – Understanding Deworming Impacts on Education (7/24/2015)
- Berger (GiveWell blog) – New deworming reanalyses and Cochrane review (7/24/2015)
- Özler (Development Impact blog) – Worm Wars: A Review of the Reanalysis of Miguel and Kremer’s Deworming Study (7/24/2015)
- Blattman (Columbia U) – The 10 things I learned in the trenches of the Worm Wars (7/24/2015)
- Taub (Vox.com) – This academic debate about worms has an important lesson for the future of global poverty (7/27/2015)
- Belluz (Vox.com) – Worm wars: The fight tearing apart the global health community, explained (7/28/2015)
- Kremer & Miguel (Thomson Reuters blog) -- The scientific case for deworming children (7/29/2015)
- Clemens & Sandefur (Center for Global Development) – Mapping the Worm Wars: What the Public Should Take Away from the Scientific Debate about Mass Deworming (7/30/2015)
- Humphreys (Columbia U) -- More on worms (8/1/2015)
- Gertler (UC Berkeley) – Good Science Gone Wrong? (8/3/2015)
- Harford (Oxford) – Worming Our Way to the Truth: Why Does Such a Large Policy Push Need to be Based on a Handful of Clinical Trials? (8/4/2015)
- Leach (Guardian) -- Explainer: Where were you in the #wormwars? (8/5/2015)
- Duflo & Karlan (IPA blog) -- Deworming: An informed debate requires a careful look at the data (8/6/2015)
- White (Linked In) -- What analysis is needed to settle the worm wars? (8/6/2015)
- An Open Letter: The Case for Deworming Children (8/8/2015), signed by a variety of organizations and individuals
- Baird, Hamory Hicks, Kremer, & Miguel (VoxEU) -- Mass deworming: (Still) a best buy for international development (8/11/2015)
- The Economist -- How to debunk a study (8/11/2015)
- BBC Radio -- More or Less: Behind the Stats, hosted by Tim Harford (8/14/2015) - from 7:45 to 16:36 in the audio segment
- Baird (World Economic Forum) -- What are the economic benefits of mass deworming of children? (8/14/2015)
- World Health Organization (WHO) updated recommendations (8/17/2015) versus one year earlier: Cites new Cochran study but recommends more expansive treatment
- Humphreys (Columbia U) -- What Has Been Learned from the Deworming Replications: A Nonpartisan View (8/18/2015)
- Belluz (Vox.com) -- The author of a contentious study on deworming finally speaks out (8/18/2015) -- Interview with Michael Kremer, co-author of the original study
- Gelman (AndrewGelman.com) -- Macartan Humphreys on the Worm Wars (8/18/2015)
- McDonald (BBC News) -- Is it worth treating everyone who might get worms? (8/19/2015)
- Mwandawiro (Gates Foundation blog) -- A New Perspective on the War on Worms (9/1/2015)
- Aiken & Davey (video of presentation at London School of Hygiene and Tropical Medicine) -- Results Reconsidered: Challenges of Re-Analyzing a Stepped-Wedge Trial with Health and Educational Outcomes (9/22/2015)
- Healthcare Triage video -- Replication, Re-Analysis, and Worm Wars (9/29/2015)
- Hicks, Kremer, & Miguel in PLOS Neglected Tropical Diseases -- The Case for Mass Treatment of Intestinal Helminths in Endemic Areas (10/22/2015)
- Walson editorial in PLOS Neglected Tropical Diseases -- Don't Shoot the Messenger (10/22/2015)
- Montresor et al in PLOS Neglected Tropical Diseases -- Methodological Bias Can Lead the Cochrane Collaboration to Irrelevance in Public Health Decision-Making (10/22/2015)
- de Silva et al in PLOS Neglected Tropical Diseases -- Cochrane Reviews on Deworming and the Right to a Healthy, Worm-Free Life (10/22/2015)
- Miguel, Kremer, & Hicks -- Comment on Macartan Humphreys’ and Other Recent Discussions of the Miguel and Kremer (2004) Study (12/21/2015)
Key research documents for the last round of the Worm Wars (i.e., when the draft replications emerged, in 2014)
- Pre-published Replication plan for Aiken et al. and Davey et al.
- Aiken et al. -- Reanalysis of health and educational impacts of a school-based deworming program in western Kenya -- Part 1: pure replication (this is the original version of the replication, published on the 3ie website (10/2014)
- Hamory Hicks, Kremer, & Miguel -- Estimating deworming school participation impacts and externalities in Kenya: A Comment on Aiken et al. (2014), response by the original authors
- Davey et al. -- Reanalysis of health and educational impacts of a school-based deworming program in western Kenya -- Part 2: alternative analyses (again, original version published on 3ie website (12/2014)
- Hamory Hicks, Kremer, & Miguel -- Estimating deworming school participation impacts in Kenya: A Comment on Aiken et al. (2014b), response by the original authors
- Data to replicate Miguel & Kremer (the original paper) -- scroll down the page to "Supplementary Materials and Data"
- Data to replicate Davey et al. (the replication)
- Ahuja et al. (and the et al. includes Kremer and Miguel) – When Should Governments Subsidize Health? The Case of Mass Deworming, World Bank Economic Review (2015)
- Baird et al. – Worms at work: Long-run impacts of a child health investment, working paper (2015)
- Bleakley – Disease and Development: Evidence from Hookworm Eradication in the American South, Quarterly Journal of Economics (2007)
- Croke – The long run effects of early childhood deworming on literacy and numeracy: Evidence from Uganda, working paper (2014)
- Ozier – Exploiting externalities to estimate the long-term effects of early childhood deworming, World Bank Policy Research Working Paper 7052 (2015)
Let me know what I’m missing; I am updating this post.
Note: The image at the top of this blog post was created by me using Wordle.net, using the content of many of the articles linked to above.
You might want to add the link for the lecture that the replication authors did at LSHTM to talk about the replication project : http://tinyurl.com/ogpvw6b
Good suggestion; I have added the video link.
Thank you! I spent all day going through the links... also, it was very useful to get my hands on the STATA dataset.
Hi, David! Happy to read your article here.
All the best to you.
Figueiró.
The replication team has posted their data/code: http://datacompass.lshtm.ac.uk/32/
Thanks! I added the data links for both the original paper and the replication papers on Friday. It's great that they are both available.
Hi there - I notice that you're missing the replication authors' response to HHKM's response: 'Authors’ Response to: Deworming externalities and school impacts in Kenya' http://ije.oxfordjournals.org/content/early/2015/07/25/ije.dyv130.full?…
Quite an important addition, I think.
Agreed and added. Thank you!
Thanks for this comprehensive review. If after going through all this material someone still wanted more Worm War I recommend going back to one of the most entertaining videogames ever published (back in 1995...) http://www.gog.com/game/worms_united
The title of this post would be perfect for a new release:)
The dataset and code for the Worms (2004) are available for download
doi:10.7910/DVN/28038
The code for 2015 replication studies are not available. Are the authors planning on placing it in the public domain?
The data are available and I've added links to the post above.
That's a great question for the authors of the study. I'd love to know the answer.
Angus Deaton writing in his 'Instruments of development: Randomization in the tropics'(2009) had an interesting tidbit that foreshadows the Worm fracas:
"the point of the RCT is less to show that deworming medicines are effective, but because children infect one another, that school-based treatment is more effective than individual treatment. As befits a paper that aims to change methods as well as practice, there is emphasis on the virtues of randomization, and the word “random” or its derivatives appears some 60 times in the paper. But the actual method of randomization is not precisely described in the paper, and private communication with Michael Kremer has confirmed that, in fact, the LOCAL PARTNERS WOULD NOT PERMIT THE USE OF RANDOM NUMBERS for assignment, so that the assignment of schools to three groups was done in alphabetical order, as in Albert to group 1, Alfred to group 2, Bob to group 3, Charles to group 1 again, David to group 2, and so on. ALPHABETIZATION, NOT RANDOMIZATION, was also used in the experiment"
Should this Alphabetization matter? Deaton elaborates on this as well:
"“Resources are often allocated alphabetically, because that is how many lists are presented, and it is easy to imagine that the Kenyan government or local NGOs, like Miguel and Kremer, used the alphabetical list to prioritize projects or funding. If so, schools higher in the alphabet are systematically different, and this difference will be inherited in an attenuated form by the three groups. Indeed, this sort of contamination is described by Cox (1958, 74–5) who explicitly warns against this sort of convenient design.”