I recently stumbled upon a TED talk given by Alex Laskey, the founder of Opower, a company using data to affect the behavior of energy consumers. The gist of his talk is that a small behavioral change can have large effects on overall consumption. It reminded me of the debate in impact evaluation (IE) and whether IE asks central or peripheral questions. In the case of energy, very little evidence exists regarding the impact of energy on the environment and the economy.
I recently was thinking about what impact evaluations in development can tell us about poverty reduction. On one level this is a ridiculous question. Most of the impact evaluations out there are designed to look at interventions to improve people's lives and the work is done in developing countries, so it follows that we are making poor people's lives better, right? That's less obvious.
In 2010, unemployment rates for Jordanian men and women between the ages of 22 to 26 with a post-secondary degree were 19 percent and 47 percent, respectively. The transition period from graduating university to stable employment for youth who do not immediately find a job is 33 months on average. This problem of educated unemployed is pervasive in many countries in MENA, and raises the question of why the labor market doesn’t clear for educated youth?
I was recently invited to speak at the biannual infrastructure retreat of the IADB and was excited to learn that they had decided to devote two days of their retreat to discussing the development of an impact evaluation (IE) program in the transport sector. This is largest sector in most development banks, yet one that has not caught the IE bug. Perhaps this is because there is a perception that IE is difficult or impossible to incorporate into transportation projects.
Public programs are designed on assumptions - nice, tidy, convenient assumptions. Then they hit the real world and very little goes as planned. The culprit, some philosophically inclined would argue, is human behavior. After all, human beings are impossible to predict. They can react in ways entirely unexpected and fairly baffling …
… until you dig deeper.
… until you dig deeper.
- On Marginal Revolution, Alex Tabarrok discusses new research on lottery-linked savings accounts.
- In the Economist, a plea for better data on time use.
I was recently working with an implementing agency to design an impact evaluation and we were having trouble reaching a point where there was going to be a viable impact evaluation that answered big questions about the efficacy of the intervention. Looking back, part of the problem was that this agency was the implementer, not the funder. And they were paid by the funder based on reaching a certain number of people and having those people participate in the program.
A good regression-discontinuity can be a beautiful thing, as Dave Evans illustrates in a previous post. The typical RD consists of controlling for a smooth function of the forcing variable (i.e. the score that has a cut-off where people on one side of the cut-off get the treatment, and those on the other side do not), and then looking for a discontinuity in the outcome of interest at this cut-off. A key practical problem is then how exactly to control for the forcing variable.