Syndicate content

Weekly links March 14: using evidence for policy, acting now vs analyzing, IEs for gender equity, and more…

David McKenzie's picture
  • Impact evaluation: a woman’s best friend? Marcelo Guigale and Markus discuss in the Huffington Post how impact evaluations can help progress towards gender equity, summarizing a variety of different studies on what works and what doesn’t to help women.
  • Jeannie Annan on the tension between action and analysis in humanitarian work: “We are plagued with this tension in many of our projects. Needs are enormous and unexpressed (or trampled) rights are rampant. If we sit and wait for the evidence, children may miss their only chance to go to school; women may have nowhere to turn when they experience violence; and countless people may have no place to go for health care, food and shelter.  Yet we have to check our urgent need to act with critical analysis—of the context, of the evidence of what works, of what the unintended consequences might be—in order to make the best choices of how to act.” This is the first in what promises to be an interesting discussion on a new blog from the International Rescue Committee.
  • Give Directly, not so fast?: on the Stanford Social Innovation Review, interesting discussion of whether this (or Chris Blattman’s work in Uganda) really generate a good bang for the donor buck. Their metric is the increase in income generated from this money, so ignores any gains from just consuming (or investing in long-term payoff investments like education). Update: Chris Blattman responds.
  • Lant Pritchett on the CGD blog has a strong challenge for impact evaluationhow to do a better job showing that it is not only influencing policy (tough enough for us to do), but causing policy changes which lead to better development outcomes  “The claim that attracted resources and support from development organizations and attention from the press was the claim that “rigorous” evidence from these RCTs could, should, and would produce better development projects and policies and, hence, ultimately better outcomes for human beings….The argument that RCTs would be more than a tiny component of the overall process of improving development outcomes seems, even now, 10 years in, at best not provable and at worst not very likely”. Of course replace RCTs with “any type of impact evaluation whatsoever” or even with “economics” and his argument doesn’t change much…Chris Blattman also offers his response.
  • On a related note, Heather Lanthorn and Suvojit Chattopadhyay discuss how evidence can get used in policymaking: “If there is a (potentially imposed) desire to commission rigorous evidence, one might assume there is genuine equipoise (or uncertainty, also here) about the efficacy, effectiveness or cost-effectiveness of a policy/program. Yet many talks about early buy-in are actually about the program and the potential to validate a flagship programme and justify related spending through evaluation” and in part 2 discuss what does influence decision-making at present “Participants at a 2012 conference on evidence-based policy-making generally agreed that “evidence is a relatively minor factor in most policy maker’s decision making” and that “many other factors” influence the decisions made.
  • In this week’s unsurprising findings: RCT finds that members of Congress are more likely to agree to meet with local campaign donors rather than local constituents.
  • Tips for Economists – a comprehensive series of links to guide you at the various stages from applying to a PhD program, how to proceed through your PhD, some various software tips, how to come up with research ideas, how to write results up, how to present them, how to apply for grants etc. They have collected tips from around the web, including several of our Development Impact posts, along with tips from a number of more famous economists.


I don't think one could replace "economics" and make the claim that it did not play more than a trivial role in cross-national differences in levels or growth rates of GDP per capita or improvement in human well being. I think there is a pretty good case to be made that poverty fell in Vietnam because the overall economy grew very fast and that the overall economy grew very fast because the government adopted good versus bad policies and that the government did this because they were convinced by economics. Similarly for massive poverty reductions in China and in Indonesia. These arguments might be wrong but they can at least be plausibly made with facts. There is nothing similar for RCTs.

I think it is still pretty hard to "prove" that economic theories directly led to policy changes that greatly improved human welfare in many countries over the past 20 years. One can come up with case studies and examples, and perhaps think of specific types of policies (e.g. monetary policy) where the links are a bit stronger. But 1) we know how hard it is to measure anything causal at the macro level; and 2) it is not enough to point to a couple of case studies and argue that "good" vs "bad" policies were the cause, without considering other countries which tried to implement the same polices and didn't have the same success.

But I think it is also clear that thinking about the links from evidence to policy change is something that it would be great to see much more research about. We know from debates on topics like immigration or trade that even when there is some agreement about a range of evidence, tracing linkages from this to particular policy decisions is very very hard.

Add new comment