Syndicate content

Dean Karlan’s new book: RCTs – this time it’s personal!

David McKenzie's picture

More than Good Intentions: How a new economics is helping to solve global poverty is a personalized helicopter tour of many recent randomized controlled trials (RCTs) in developing countries. It is written by Dean Karlan, who has been a researcher in many of these experiments, and Jacob Appel, who worked for Dean in implementing many of these experiments in Ghana. The personal stories of people Jake has come across on his travels are used by the authors as a lens to illustrate broader points in the constraints that keep people poor and the policies that might overcome these constraints.

>Hence we are introduced to some of the issues that various experiments are seeking to address through people like Mercy, the Makola market vendor and microfinance client who has covered for others in her group not repaying one time too many; Vijaya, the flower garland maker in Chennai who lacks a mechanism to save; Elizabeth, who goes to a herbalist instead of a doctor when she has a broken bone; and Erlyn, the sari sari store owner in the Philippines who store is described thusly:

"Above the counter in her sari sari store a great waterfall of pork rinds was frozen in midcascade, suspended in plastic bags and in bandoliers of foil pouches, hung from the lintel with binder clips and fishing line. The other popular item was Tang, and it was also well-represented, a kaleidoscope of colored sachets strewn among the shelves".

Writing like this makes clear this is no dry academic account, but rather a successful attempt to communicate the results of a new wave of studies in development to a broader audience. If you work in randomized impact evaluations, this is a book you can recommend to your friends or family to let them finally understand what is it exactly that you do. It is definitely one you should read. It does not just cover Dean’s experiments, but a range of experiments in microfinance and savings, education, health, and farming (full disclosure: about a page and a half covers some of my own work).

The book also serves as a challenge or competing view to the recent macro debate between people like Easterly, Sachs and Collier on aid effectiveness. Dean nicely summarizes his point of view on page 38:

“Addressing world poverty is a dynamic, complex problem. But we won’t solve it if we see it only as that. We need to see individuals. Individuals with different capabilities and different needs. Individuals like Vijaya…What she really needs is a way to stop her husband from drinking away the money she earns…Individuals like Elizabeth…What she really needs is better customer service from her local hospital. When we think about poverty this way, in concrete terms, we begin to see a path forward. Actually many paths forward. The possible solutions are as numerous and varied as the people they serve and the needs they address. To find them we need to think creatively, cast a wide net, and recognize that we are unlikely to find a single answer for everybody. At the same time, we must be methodical and tenacious. If a development program is supposed to help solve a specific concrete problem, let’s put it to a specific, concrete test. If it passes, great. If not, fix it or try something else. In this way, step by step, we can refine the tools we use and the ways we use them: we can make real progress against poverty”.

This is then one of the central ideas of the book – that one way to make progress against poverty is to test different programs in a range of settings using RCTs, and learn what works. The other main theme in the book comes from behavioral economics. The basic idea is that because of behavioral biases, people aren’t always making decisions that are optimal for them in the long run, and as a result, cheap “nudges” can improve lives. And so we also get a tour of commitment savings accounts, text messages which remind people to save, and hear how changing the timing of payments in a conditional cash transfer program in Colombia allowed it to improve on the much heralded Progresa/Oportunidades program.

One of the most interesting things for me to read about in the book was to hear what happened after the academic studies ended. For example, many readers may have heard of a study in India which attempted to overcome teacher absence by placing cameras in schools and getting teachers to take daily photos with the students. The book tells us that the NGO implementing this has now gone on to make this standard policy for all its schools due to results of the experiment. We likewise get to hear what came next in several other well-known experiments, although this is one area where I would have liked the book to tell us even more.

Putting a whole range of different experiments together allows us to make progress towards the bigger and more controversial question of what we are learning more generally from the new wave of randomized experiments (see also the recent Boston Review discussion). I see the glass as half-full – the results of these studies show we do know some things that work in making life a bit better for poor people- we can help them save a little more, get a bit more education, and stay a bit healthier. This is no small feat, and is important news for donors seeking to alleviate poverty. But how far does it take us toward not just poverty alleviation, but development?

I think there is still great promise there also – there are a number of ongoing and recent experiments which consider not just how to make the poor a little better off, but also how to make government work better for them, how to facilitate movements of people from rural to urban areas and from poor to rich countries, how to get SMEs and larger firms to become more productive, and how to generate more employment. Experiments can’t answer every question of interest, but More than good intentions makes a strong case for needing to learn what works whenever we can.

Comments

Submitted by Anonymous on
See here - http://sevamandir.org/education.htm - that the camera project now is implemented in 38 out of the 176 village centres by the NGO, Seva Mandir (SM). SM was an enthusiastic partner and continues to be so, for many JPAL studies. Wonder what then hinders the wider scale implementation of such a proven success? Is this an isolated case? Probably not.

I followed up on this with Esther, who was kind enough to respond that the cameras are used in all of the village centres today. So I suspect the Seva Mandir website just needs updating. Your general point is a good one though, and it would be great to hear from those involved in NGOs that have been subject to impact evaluations as to why there is sometimes not wider scale implementation. This is also something we'll try and follow up on in future blog posts - to chase up some other studies and see what happened when the paper finished. If there is a study that readers are particularly curious about what happened next, let us know and we'll see what we can find out.

Submitted by guest on
I wish this would be implemented in my country. I agree that the solution to poverty is to test different programs in a range of settings using RCTs and hope for the best that the budget for this program will not go to the politicians' pockets but instead focus on the experiments.