Although the members of the Millennium Village Project were unavailable (but have offered to hold a follow-up seminar in January), we held a seminar on the Clemens-Demombynes paper to discuss different approaches to evaluating rural development programs. My reaction to the presentation and discussion is that there really shouldn’t be a controversy here. It’s true that the original paper by the MVP, “Harvests of Development” does not have a properly specified counterfactual (what would have happened in the absence of the intervention, including the service delivery innovations of the project) and therefore does not constitute an evaluation. The Clemens-Demombynes (CD) paper attempts to specify a counterfactual by comparing the performance of MVP villages with that of neighboring and similar villages. While not perfect, this procedure meets many of the criteria for a reasonable evaluation. In fact, as we discussed at the seminar, it avoids some of the possible “contamination effects” of a seemingly more rigorous randomized evaluation.
But the point is that the CD paper shows that, relative to their counterfactual, the MVP performs well along a number of dimensions, such as improved sanitation, measles vaccination, and reducing child mortality. Not surprisingly, it does not perform well in cell phone usage in Kenya, a country where mobile phones have spread throughout the country. In short, Clemens and Demombynes have undertaken the first evaluation of the MVP. They have shown that the MVP has delivered sizeable improvements on some important development indicators in many of the villages, albeit with effects that are smaller than those described in the Harvests of Development paper. Of course, neither study answers the question of whether these gains are sustainable, or whether they could have been obtained at lower cost. These should be the subject of the next evaluation.
Join the Conversation