Syndicate content

Impact evaluation

Are Impact Evaluations Enough? The Social Observatory Approach to Doing-by-Learning

Vijayendra Rao's picture

Impact Evaluations are just one of many important tools to improve “adaptive capacity.” To improve implementation, they need to be integrated with monitoring and decision support systems, methods to understand mechanisms of change, and efforts to build feedback loops that pay attention both to everyday and long-term learning.  While there has been some scholarly writing and advocacy on this point, it has been more talk than action. 

Hammers, wrenches, and development policy

Markus Goldstein's picture
The old saw goes: when you have a hammer, everything looks like a nail.    But what if the best way to fix your broken policy is actually a bolt?   I was recently at a workshop where someone was presenting preliminary results of an evaluation cash transfer program which, while perhaps started with social protection kind of objectives in mind, actually seems to have had impacts on business creation and revenues that dwarfed your average business training program or microfinance program. 
 

The International Rescue Committee's approach to impact evaluation

Markus Goldstein's picture
Our series on institutional approaches to impact evaluation continues!    DI recently virtually sat down with Jeannie Annan, Director of Research and Evaluation at the International Rescue Committee.   
 
DI:  What is the overall approach to impact evaluation at the IRC?
JA:  We are committed to providing (or supporting) the most effective and cost-effective interventions. This means using the best available research about what works combined with understanding of the context and experience in implementation to design and deliver our programs.  

The MCC's approach to impact Evaluation

Markus Goldstein's picture
In our continuing series on discussing institutional approaches to impact evaluation, DI virtually sat down with Jack Molyneaux, Director of Independent Evaluations at the Millennium Challenge Corporation.  (Please note that these are Jack’s opinions, not that of the MCC)
 
DI:  Impact Evaluation seems to be something that's pretty important at the MCC. Can you tell us a bit about how this focus came about? 
JM: Since its inception MCC’s mandate has included demonstrating results.  Rigorous impact evaluations have always been a key component of that mandate. 

Weekly links July 12: daycare, remittances for education, opaque measurement, funding, and more…

David McKenzie's picture
  • How should we measure what is a high-income country? Martin Ravallion explains and critiques the World Bank definition on the CGD blog.
  • Aid Thoughts discusses new work on the value of daycare in Brazilian slums.
  • A new From Evidence to Policy note looks at the long-term impact of a conditional cash transfer on education in Colombia-part of the analysis uses admin data on test scores for graduating students – “students whose families received cash grants were between 4 and 8.4 percentage points more likely to graduate from high school; but Students whose families received the cash grants didn’t score higher on the national standardized achievement test given a year before graduation”.
  • Classic papers in behavioral finance summed up in a few sentences – Noah Smith gives his take on essential papers in behavioral finance.
  • On the IDB Development Effectiveness blog, Dean Yang and co-authors summarize their new study on the use of matching funds to channel remittances towards education in El Salvador.
  • Funding opportunity: The World Bank’s Strategic Impact Evaluation Fund (SIEF) has a new call for proposals for work on basic education, water and sanitation, early childhood development, and health systems. Details here.
  • Funding opportunity: 3ie has funding available under an agricultural innovation thematic window. This grant window will fund up to 16 new impact evaluations of interventions in the areas of knowledge transfer, contractual arrangements, adoption, and soil health
  • Funding opportunity: (Not just for impact evaluations) IZA and DFID are now accepting applications for funding in Phase III of the Growth and Labor Markets in Low Income Countries (GLM | LIC) program.  This will fund work on 1. Growth and labor market outcomes, 2. Active labor market policies, 3. Labor market institutions, 4. Migration and labor markets, 5. Gender and 6. Data for labor market analysis. Application materials here.

Public randomization ceremonies

Damien de Walque's picture

With Jake Robyn* and Gaston Sorgho**

Randomization might- at first – sound like a scary word for health policy makers and professionals. They read medical journals and know from their training that randomized trials are scientifically rigorous designs to evaluate the impact of a program. But their first inclination might be to prefer to have the randomized trial in somebody else’s backyard. Randomization seems politically difficult. How to explain it to the people who will have to wait for the new intervention? Will it not create a backlash with the people who are randomly assigned to the control group? How will the population be convinced that the random allocation was fair and that there were no back room deals?

Our experience in many countries is that public randomization ceremonies are an excellent platform to build support for randomization and for the entire impact evaluation process. In Cameroon, we organized public randomization ceremonies in three Regions to assign health facilities to four study groups in an impact evaluation of performance-based financing (PBF) in the health sector. Held in the regional capitals and combined with the official launching of the project in each Region, we invited representatives of each facility, district health management teams, and local government, who all took part themselves in the randomization. Each of the randomization ceremonies received close oversight from the central and regional levels of the Ministry of Public Health. This made the randomization process completely fair and transparent to all health facilities participating in the study.

Introducing the Africa Gender Innovation Lab

Markus Goldstein's picture

Today I wanted to take the opportunity to talk about a new initiative that the Africa Region and the Research Group at the World Bank are launching today.   The idea here is that we don't know enough about how to effectively address the underlying causes of gender inequality. Let me start by explaining what I mean by underlying causes.    Take the case of female farmers.    There is a lot of literature out there which shows that women have lower agricultural yields than men.   And some of it shows that this is because women have lo

It’s Not about the Technology, It’s about the People: Evaluating the Impact of ICT Programs

Shamiela Mir's picture

How can we better design ICT programs for development and evaluate their impact on improving peoples’ well-being? A new approach, the Alternative Evaluation Framework (AEF) takes into account multiple dimensions of peoples’ economic, social and political lives rather than simply focusing on access, expenditure and infrastructure of ICT tools. This new approach is presented in How-To Notes, Valuing Information: A Framework for Evaluating the Impact of ICT Programs, authored by Bjorn-Soren Gigler, a Senior Governance Specialist at the World Bank Institute’s Innovation Practice.

When We (Rigorously) Measure Effectiveness, What Do We Find? Initial Results from an Oxfam Experiment

Duncan Green's picture

Guest post from ace evaluator Dr Karl Hughes (right, in the field. Literally.)

Just over a year ago now, I wrote a blog featured on FP2P – Can we demonstrate effectiveness without bankrupting our NGO and/or becoming a randomista? – about Oxfam’s attempt to up its game in understanding and demonstrating its effectiveness.  Here, I outlined our ambitious plan of ‘randomly selecting and then evaluating, using relatively rigorous methods by NGO standards, 40-ish mature interventions in various thematic areas’.  We have dubbed these ‘effectiveness reviews’.  Given that most NGOs are currently grappling with how to credibly demonstrate their effectiveness, our ‘global experiment’ has grabbed the attention of some eminent bloggers (see William Savedoff’s post for a recent example).  Now I’m back with an update.


Pages