Impact Evaluations
Syndicate content

Impact evaluation

Public randomization ceremonies

Damien de Walque's picture

With Jake Robyn* and Gaston Sorgho**

Randomization might- at first – sound like a scary word for health policy makers and professionals. They read medical journals and know from their training that randomized trials are scientifically rigorous designs to evaluate the impact of a program. But their first inclination might be to prefer to have the randomized trial in somebody else’s backyard. Randomization seems politically difficult. How to explain it to the people who will have to wait for the new intervention? Will it not create a backlash with the people who are randomly assigned to the control group? How will the population be convinced that the random allocation was fair and that there were no back room deals?

Our experience in many countries is that public randomization ceremonies are an excellent platform to build support for randomization and for the entire impact evaluation process. In Cameroon, we organized public randomization ceremonies in three Regions to assign health facilities to four study groups in an impact evaluation of performance-based financing (PBF) in the health sector. Held in the regional capitals and combined with the official launching of the project in each Region, we invited representatives of each facility, district health management teams, and local government, who all took part themselves in the randomization. Each of the randomization ceremonies received close oversight from the central and regional levels of the Ministry of Public Health. This made the randomization process completely fair and transparent to all health facilities participating in the study.

Introducing the Africa Gender Innovation Lab

Markus Goldstein's picture

Today I wanted to take the opportunity to talk about a new initiative that the Africa Region and the Research Group at the World Bank are launching today.   The idea here is that we don't know enough about how to effectively address the underlying causes of gender inequality. Let me start by explaining what I mean by underlying causes.    Take the case of female farmers.    There is a lot of literature out there which shows that women have lower agricultural yields than men.   And some of it shows that this is because women have lo

The Tao of Impact Evaluation

Markus Goldstein's picture

Is in danger of being messed up.   Here is why:   There are two fundamental reasons for doing impact evaluation: learning and judgment.   Judgment is simple – thumbs up, thumbs down: program continues or not.   Learning is more amorphous – we do impact evaluation to see if a project works, but we try and build in as many ways to understand the results as possible, maybe do a couple of treatment arms so we see what works better than what. In learning evaluations, real failure is a lack of statistical power, more so than the program working or

DFID's Approach to Impact Evaluation - Part II

Markus Goldstein's picture

As part of a new series looking how institutions are approaching impact evaluation, DI virtually sat down with Nick York, Head of Evaluation and Gail Marzetti, Deputy Head, Research and Evidence Division.   For Part I of this series, see yesterday’s post. Today we focus on DFID’s funding for research and impact evaluation.  

Notes from the field: Making the most of monitoring data

Markus Goldstein's picture

I am in the midst of a trip working on impact evaluations in Ghana and Tanzania and these have really brought home the potential and pitfalls of working with program’s monitoring data.  

In many evaluations, the promise is significant. In some cases, you can even do the whole impact evaluation with program monitoring data (for example when a specific intervention is tried out with a subset of a program’s clients).  However, in most cases a combination of monitoring and survey data is required.

Is our columnists learning?

Markus Goldstein's picture

In a New York Times column last Friday David Brooks discussed a book by Jim Manzi, and extolled the idea of randomized field trials as a way for the US to make better policies. 

While it’s nice to welcome Citizen Brooks into the fold, there are a couple of points in his article worth exploring a bit.  

Notes from the field: Sometimes you’re the windshield, sometimes you’re the bug

Markus Goldstein's picture

So this past week I was in Ghana following up on some of the projects I am working on there with one of my colleagues.   We were designing an agricultural impact evaluation with some of our counterparts, following up on the analysis of the second round of a land tenure impact evaluation and a financial literacy intervention, and exploring the possibility of some work in the rural financial sector.   In no particular order, here are some of the things I learned and some things I am still wondering about:

    Notes from the field: the usefulness of early workshops

    Markus Goldstein's picture

    One of the things I learned from other folks at the Bank I work with is the usefulness of doing a workshop early in the early design of an impact evaluation to bring the project and the impact evaluation team together to hammer out design.   With one of my colleagues, I did one of these during my recent trip to Ethiopia and a bunch of things stuck out.  

    Pages