Guest post by Dean Karlan and Jacob Appel
Dean has failed again! Dean and Jacob are kicking off our series on learning from failure by contributing a case that wasn’t in the book.
I. Background + Motivation
Recent changes in the aid landscape have allowed donors to support small, nimble organizations that can identify and address local needs. However, many have lamented the difficulties of monitoring the effectiveness of local organizations. At the same time as donors become more involved, the World Bank has called for greater “beneficiary control,” or more direct input from people receiving development services.
While attempts have been made to increase the accountability of non-profits, little research addresses whether doing so actually encourages donors to give more or to continue supporting the same projects. On the contrary it may be that lack of accountability provides donors with a convenient excuse for not giving. It could be that donors give the same amount even with greater accountability. Furthermore, little research indicates whether increased transparency and accountability would provide incentives for organizations to be more effective in providing services. Rigorous research will help determine the impact of increasing accountability, both on the behavior of donors and on the behavior of organizations working in the field.
Guest post by Dean Karlan and Jacob Appel
Dean Karlan and Jacob Appel have a new book out called Failing in the Field: What we can learn when field research goes wrong. It is intended to highlight research failures and what we can learn from them, sharing stories that otherwise might otherwise be told only over a drink at the end of a conference, if at all. It draws on a number of Dean’s own studies, as well as those of several other researchers who have shared stories and lessons. The book is a good short read (I finished it in an hour), and definitely worth the time for anyone involved in collecting field data or running an experiment.
Heckman turned one of his lectures from last year into an NBER WP titled "Capabilities and Skills." Looks really interesting - here's a quote from the abstract: "We address measurement problems common to both the economics of human development and the capability approach. The economics of human development analyzes the dynamics of preference formation, but is silent about which preferences should be used to evaluate alternative policies. This is both a strength and a limitation of the approach."
Advances in Econometrics has a special issue on Regression Discontinuity Design, including many papers by prominent statisticians and econometricians in the field and edited by Cattaneo and Escanciano.
Do poor people want more redistributive programs and less public goods? Latest issue of the Journal of the European Economic Association has a paper by Bursztyn that challenges elite capture as the explanation for low levels of investment in public education. Here is the abstract: "A large literature has emphasized elite capture of democratic institutions as the explanation for the low levels of spending on public education in many low-income democracies. This paper provides an alternative to that longstanding hypothesis. Motivated by new cross-country facts and evidence from Brazilian municipalities, we hypothesize that many democratic developing countries might invest less in public education spending because poor decisive voters prefer the government to allocate resources elsewhere. One possible explanation is that low-income voters could instead favor redistributive programs that increase their incomes in the short run, such as cash transfers. To test for this possibility, we design and implement an experimental survey and an incentivized choice experiment in Brazil. The findings from both interventions support our hypothesis."
My brilliant former research assistant Utz Pape has a blog post titled: "What did we learn from real-time tracking of market prices in South Sudan?" Read it if you're into innovative use of technology in development.
Private Enterprise Development in Low-Income Countries (PEDL), a joint research initiative of the Centre for Economic Policy Research (CEPR) and the Department For International Development (DFID), is offering a competitive research grants scheme for projects related to the behaviour of firms in Low-Income Countries (LICs) that aim to better understand what determines the strength of market forces driving efficiency in these countries. Round 21 of their new Exploratory Research Grants is now open:
The PEDL webpage:http://pedl.cepr.org/
Information on how to apply: http://pedl.cepr.org/content/exploratory-research-grants-0
- weekly links
Practical advice on robust standard errors in (not so small) samples: Imbens and Kolesár have an old working paper just published in REStat that tells you to do three things:
Solomon Hsiang and Nitin Sekar respond to the guest post by Quy-Toan Do and co-authors which had re-analyzed their data to question whether a one-time legal sale of ivory had increased elephant poaching. They state “Their claims are based on a large number of statistical, coding, and inferential errors. When we correct their analysis, we find that our original results hold for sites that report a large number of total carcasses; and the possibility that our findings are artifacts of the data-generating process that DLM propose is extremely rare under any plausible set of assumptions”.
- We screwed up by hosting this guest post without checking that Do and co-authors had shared it with the original co-authors and had given them a chance to respond.
- We do believe that blogs have an important role to play in discussing research (see also Andrew Gelman on this), but think Uri Simonsohn’s piece this week on how to civilly argue with someone else’s analysis has good practice ideas for both social media and refereeing – with sharing the discussion with authors beforehand when re-analysis is done being good practice. We will try to adhere to this better in the future.
- We are waiting to see whether Do and co-authors have any further word, and plan on posting only one more summary on this after making sure both sides have iterated. We plan to avoid Elephant wars since worm wars were enough.
- In somewhat related news, Dana Carney shows how to gracefully accept and respond to criticism over your earlier work.
or, why we need more systematic (and simply more) reporting on the nature of interventions
The hope. Last year, we reviewed six reviews of what interventions work to improve learning. One promising area of overlap across reviews had to do with training teachers who were already on the job (i.e., in-service teacher training or teacher professional development). Specifically, we proposed that “individualized, repeated teacher training, associated with a specific method of task” was associated with learning gains.
- The second edition of the book Impact Evaluation in Practice by Paul Gertler, Sebastian Martinez, Patrick Premand, Laura Rawlings and Christel Vermeersch is now available. For free online! “The updated version covers the newest techniques for evaluating programs and includes state-of-the-art implementation advice, as well as an expanded set of examples and case studies that draw on recent development challenges. It also includes new material on research ethics and partnerships to conduct impact evaluation.”
- Interesting Priceonomics piece on R.A. Fisher, and how he fought against the idea that smoking causes cancer
- Oxfam blog post on power calculations for propensity score matching
The importance of airlines for research and growth:
- VoxEU piece on work showing that when Southwest airlines opens up routes between U.S. cities, more scientific collaboration in Chemistry occurs (diff-in-diff) and NPR story on this
- New working paper by Campante and Yanagizawa-Drott uses discontinuity in connectiveness of cities at 6,000 miles to show air links have positive impact on economic activity by inducing links between businesses. The Kennedy school provides a media summary here.