When state institutions find it a challenge to deliver services in under-resourced areas, its common for policy makers to consider leveraging existing local non-state capacity to help. This involvement of NGOs or CBOs is meant to supplement the state as service provider but a recent paper by Ashis Das, Eeshani Kandpal, and me demonstrates possible pitfalls with this extension approach. Just as implementation capacity of governments is a key determinant of government program performance, NGO capacity is a key determinant of NGO performance and under-resourced areas are likely to contain under-resourced local organizations. We find this to be the case in our study context of malaria control in endemic regions of India. Besides highlighting this challenge, our results also highlight the difficulties that small-scale evaluations present to the generalizability of findings, especially those implemented by non-state actors. Implementation capacity can be a key confounder of generalizability and it is not often measured or even discussed the current practice of impact evaluation needs to think harder about measures that capture implementation capacity in order to generalize IE results to other contexts.
- Chris Blattman on using qualitative measurement to validate quantitative measures of risky behaviors.
- Rachel Glennerster on individual vs community incentives for service provision.
I have been planning to write a blog post for a few months now. Many of you may know how procrastination works! This changed after I attended the 4th World Bank Conference on Equity on May 29, 2014. This year, the focus was on Aspirations, Poverty and Inequality. Listening to the likes of Debraj Ray and Glenn Loury is always a treat. Ray talked about the idea of aspirational gaps, defined as the difference between a person's contemporaneous standard of living and the standard of living she or he aspires to. Ray stressed that getting aspirations right is important: too low and people will not take action; too high and it might lead to frustration.
But what happens when you get it right? Let me tell you about a project in Nicaragua that sheds some insights on how aspirations can enhance the impacts of a program.
Impact Evaluations are just one of many important tools to improve “adaptive capacity.” To improve implementation, they need to be integrated with monitoring and decision support systems, methods to understand mechanisms of change, and efforts to build feedback loops that pay attention both to everyday and long-term learning. While there has been some scholarly writing and advocacy on this point, it has been more talk than action.
When I start working on a new impact evaluation, I often begin with a workshop in the country where the study will be conducted. The workshop brings together government officials, both at the central level and from the regions and provinces where the intervention will take place, other stakeholders such as NGOs or other UN organizations, and representatives of the research institution that will implement the survey. Part of the workshop is devoted to teaching or refreshing memories about evaluation techniques. This usually includes a section on randomization which we try to make interactive by doing a randomization game with the participants.
- Visa lottery evidence on the impact of high-skilled immigration on jobs for US workers – Vox covers new work by Giovanni Peri and co-authors – they find cities which were unlucky in the H1-B lottery experience slower job growth for American workers in the tech sector without college degrees,