The New York Times recently had a piece on the retraction and re-issuance of a study in Spain based on a randomized trial of the Mediterranean Diet’s effect on heart disease. The original study was meant to be an individualized random assignment of 7,447 people aged 55 to 80 to one of three different diets – a control diet (advice to just reduce fat content), or two variants of the Mediterranean Diet (in which they were given free olive oil or free nuts). The study was originally published in the New England Journal of Medicine (NEJM) in 2013. The authors then appear to have been surprised to find their study on a list of suspicious trials. There are several parts to this story I thought would be of interest for doing impact evaluations in development, which I discuss below.
One of the standard defenses of an RCT proposal to a skeptic is to invoke budget and implementation capacity constraints and argue that since not everyone will get the desired treatment (at least initially), the fairest way would be to randomly allocate treatment among the target population. While this is true, it is also possible to take into consideration the maximization of participants’ welfare and incorporate their preferences and expected responses to treatment into account while designing an RCT that still satisfies the aims of the researcher (identify unbiased treatment effects with sufficient precision). A recent paper by Yusuke Narita seems to make significant headway in this direction for development economists to take notice.
A recent paper in Lancet Global Health found that generous conditional cash transfers to female secondary school students had no effect on their school attendance, dropout rates, HIV incidence, or HSV-2 (herpes simplex virus – type 2) incidence. What happened?
There is a minor buzz this week in Twitter and the development economics blogosphere about a paper (posted on the CSAE 2012 Conference website) that discusses a double blind experiment of providing different seeds of cowpeas to farmers in Tanzania.
Following on David’s rant on external validity yesterday, which turned out to be quite popular, I decided to keep the thread going. Despite the fact that the debate is painted in ‘either/or’ terms, my feeling is that there are things that careful researchers/evaluators can do to improve the external validity of their studies.