The World Region
(based on discussions with Chico Ferreira, Ambar Narayan, Carolina Sanchez and especially, Aaditya Mattoo)
This is part I of a three-part series.
Last month, NIAID released news that treating HIV-infected partners in mostly heterosexual HIV-discordant couples at 13 sites around the world reduced HIV transmissi
As a fair number of impact evaluations I work on are programs designed by governments or NGOs, I often initially have to have a tricky discussion when it comes time to do the power calculations to design the impact evaluation. The subject of this conversation is the anticipated effect size. This is a key parameter – if it’s too optimistic you run the risk of an impact evaluation with no effect even when the program had worked to some (lesser) degree, if it’s too pessimistic, then you are wasting money and people’s time in your survey.
My last post discussed an example of a system intervention (improvements to the pharmaceutical supply chain) and the not uncommon inferential challenge of low power from relatively few units of observation.
More than Good Intentions: How a new economics is helping to solve global poverty is a personalized helicopter tour of many recent randomized controlled trials (RCTs) in developing countries. It is written by Dean Karlan, who has been a researcher in many of these experiments, and Jacob Appel, who worked for Dean in implementing many of these experiments in Ghana.
A quick look at the burgeoning literature on policy evaluations will reveal a preponderance of evaluations of demand side schemes such as conditional cash transfers. There is an obvious reason for this beyond the promise that such interventions hold: the technology of treatment allows for large sample randomized evaluations, either at the household or community/village level. As long as financing is sufficient to sample an adequate number of study units, study power will not be a concern.
The March 2011 issue of the Harvard Business Review has “a step-by-step guide to smart business experiments” by Eric Anderson and Duncan Simester, two marketing professors who have done a number of experiments with large firms in the U.S. Their bottom line message for businesses is:
Welcome to our new blog about impact evaluation. The number of impact evaluations both within and outside the World Bank have increased dramatically in recent years (see Figure below). The World Bank’s DIME initiative reports the number of active impact evaluations at the World Bank has increased from a couple of dozen in 2004 to over 250 in 2010. As a result, there are more producers and more consumers of impact evaluations than ever before – and more questions about how to conduct evaluations, what we are learning from these evaluations, and what can be done to learn more from them.
This blog has been set up by members of the Development Research Group at the World Bank to provide a forum for discussing these issues.
(Source: DIME. Figure shows impact evaluations by World Bank region: SAR = South Asia, MNA = Middle East and North Africa, LAC= Latin America and the Caribbean, ECA = Europe and Central Asia, EAP = East Asia and the Pacific, AFR = Sub-Saharan Africa)
Our goal is to cover a broad range of issues relating to impact evaluations, including:
- Summaries, critiques, and discussions of new research papers
- Discussions of ongoing and planned evaluations
- Methodological issues in doing experimental and non-experimental evaluations
Stories and puzzles from fieldwork
We want this blog to be something that is useful to both producers and consumers of impact evaluations, which can provide an opportunity for serious discussion and sharing of experiences. We hope to have a range of guest bloggers from both inside and outside the World Bank. We encourage all readers to not only participate in the discussions by commenting on blog posts, but also sending ideas and materials for blogs, as well as for guest blogging.
Please let us know in the comments any ideas you have for pressing issues to discuss, or topics you would like to see covered in future posts.