Estimating the direct and indirect benefits of transport projects remains difficult. Only a handful of rigorous impact evaluations have been done as the methodologies are technically and financially demanding. There are also differences between the impact of rural and urban projects that need to be carefully anticipated and evaluated.
Can we simplify the methodologies?
Despite the Bank’s rich experience with transport development projects, it remains quite difficult to fully capture the direct and indirect effects of improved transport connectivity and mobility on poverty outcomes. There are many statistical problems that come with impact evaluation. Chief among them, surveys must be carefully designed to avoid some of the pitfalls that usually hinder the evaluation of transport projects (sample bias, timeline, direct vs. indirect effects, issues with control group selection, etc.).
Impact evaluation typically requires comparing groups that have similar characteristics but one is located in the area of a project (treatment group), therefore it is likely to be affected by the project implementation, while the other group is not (control group). Ideally, both groups must be randomly selected and sufficiently large to minimize sample bias. In the majority of road transport projects, the reality is that it is difficult to identify control groups to properly evaluate the direct and indirect impact of road transport improvements. Also, road projects take a long time to be implemented and it is difficult to monitor the effects for the duration of a project on both control and treatment groups. Statistical and econometric tools can be used to compensate for methodological shortcomings but they still require the use of significant resources and knowhow to be done in a systematic and successful manner.
When it comes to helping young women in Africa with both economic and social opportunity, what does the evidence tell us? Broadcaster Georges Collinet sat down with researchers and policymakers to discuss the hard evidence behind two programs that have succeeded in giving girls a better chance at getting started in their adult lives.
Impact Evaluations are just one of many important tools to improve “adaptive capacity.” To improve implementation, they need to be integrated with monitoring and decision support systems, methods to understand mechanisms of change, and efforts to build feedback loops that pay attention both to everyday and long-term learning. While there has been some scholarly writing and advocacy on this point, it has been more talk than action.
DI: What is the overall approach to impact evaluation at the IRC?
JA: We are committed to providing (or supporting) the most effective and cost-effective interventions. This means using the best available research about what works combined with understanding of the context and experience in implementation to design and deliver our programs.
DI: Impact Evaluation seems to be something that's pretty important at the MCC. Can you tell us a bit about how this focus came about?
JM: Since its inception MCC’s mandate has included demonstrating results. Rigorous impact evaluations have always been a key component of that mandate.
- How should we measure what is a high-income country? Martin Ravallion explains and critiques the World Bank definition on the CGD blog.
- Aid Thoughts discusses new work on the value of daycare in Brazilian slums.
- A new From Evidence to Policy note looks at the long-term impact of a conditional cash transfer on education in Colombia-part of the analysis uses admin data on test scores for graduating students – “students whose families received cash grants were between 4 and 8.4 percentage points more likely to graduate from high school; but Students whose families received the cash grants didn’t score higher on the national standardized achievement test given a year before graduation”.
- Classic papers in behavioral finance summed up in a few sentences – Noah Smith gives his take on essential papers in behavioral finance.
- On the IDB Development Effectiveness blog, Dean Yang and co-authors summarize their new study on the use of matching funds to channel remittances towards education in El Salvador.
- Funding opportunity: The World Bank’s Strategic Impact Evaluation Fund (SIEF) has a new call for proposals for work on basic education, water and sanitation, early childhood development, and health systems. Details here.
- Funding opportunity: 3ie has funding available under an agricultural innovation thematic window. This grant window will fund up to 16 new impact evaluations of interventions in the areas of knowledge transfer, contractual arrangements, adoption, and soil health
- Funding opportunity: (Not just for impact evaluations) IZA and DFID are now accepting applications for funding in Phase III of the Growth and Labor Markets in Low Income Countries (GLM | LIC) program. This will fund work on 1. Growth and labor market outcomes, 2. Active labor market policies, 3. Labor market institutions, 4. Migration and labor markets, 5. Gender and 6. Data for labor market analysis. Application materials here.