My last post discussed an example of a system intervention (improvements to the pharmaceutical supply chain) and the not uncommon inferential challenge of low power from relatively few units of observation.
More than Good Intentions: How a new economics is helping to solve global poverty is a personalized helicopter tour of many recent randomized controlled trials (RCTs) in developing countries. It is written by Dean Karlan, who has been a researcher in many of these experiments, and Jacob Appel, who worked for Dean in implementing many of these experiments in Ghana.
Update: Lant Pritchett has kindly responded to my invitation and posted his thoughts: "No need for unmet need." Check out the comments section.
A quick look at the burgeoning literature on policy evaluations will reveal a preponderance of evaluations of demand side schemes such as conditional cash transfers. There is an obvious reason for this beyond the promise that such interventions hold: the technology of treatment allows for large sample randomized evaluations, either at the household or community/village level. As long as financing is sufficient to sample an adequate number of study units, study power will not be a concern.
As those of you who follow the blogosphere on development economics may have heard, Chris Blattman and Jeannie Annan had a healthy baby born this past Sunday. The mother and the baby are reported to be happy and healthy. We send Chris, Jeannie, and little Amara our best wishes.
You can read Chris' blog here.
In some joint work with an African government, my colleagues Francisco Campos, Jessica Leino and I were trying to evaluate the impacts of one of their support programs for small businesses. This service was open to anyone who contacted them, but the number of entrepreneurs who knew about the program (and hence who used it) was low. Basically, the way the program worked was that when the entrepreneur came into the office and registered for the program, the implementing agency would assess the needs of the business and then provide the entrepreneur with subsidized access to
There are now a variety of well-known experimental and non-experimental methods that economists use to learn whether a given program works or not. However, our tools for learning why or why not something works are much more limited.
The March 2011 issue of the Harvard Business Review has “a step-by-step guide to smart business experiments” by Eric Anderson and Duncan Simester, two marketing professors who have done a number of experiments with large firms in the U.S. Their bottom line message for businesses is:
Welcome to our new blog about impact evaluation. The number of impact evaluations both within and outside the World Bank have increased dramatically in recent years (see Figure below). The World Bank’s DIME initiative reports the number of active impact evaluations at the World Bank has increased from a couple of dozen in 2004 to over 250 in 2010. As a result, there are more producers and more consumers of impact evaluations than ever before – and more questions about how to conduct evaluations, what we are learning from these evaluations, and what can be done to learn more from them.
This blog has been set up by members of the Development Research Group at the World Bank to provide a forum for discussing these issues.
(Source: DIME. Figure shows impact evaluations by World Bank region: SAR = South Asia, MNA = Middle East and North Africa, LAC= Latin America and the Caribbean, ECA = Europe and Central Asia, EAP = East Asia and the Pacific, AFR = Sub-Saharan Africa)
Our goal is to cover a broad range of issues relating to impact evaluations, including:
- Summaries, critiques, and discussions of new research papers
- Discussions of ongoing and planned evaluations
- Methodological issues in doing experimental and non-experimental evaluations
Stories and puzzles from fieldwork
We want this blog to be something that is useful to both producers and consumers of impact evaluations, which can provide an opportunity for serious discussion and sharing of experiences. We hope to have a range of guest bloggers from both inside and outside the World Bank. We encourage all readers to not only participate in the discussions by commenting on blog posts, but also sending ideas and materials for blogs, as well as for guest blogging.
Please let us know in the comments any ideas you have for pressing issues to discuss, or topics you would like to see covered in future posts.