Following up on Michael’s post yesterday, I wanted to add a couple of thoughts.
Some of the more impressive results I have seen have come out of papers which take a long term view. Three examples of this are Esther Duflo’s work on school construction in Indonesia (ungated version here), Hoddinott, et. al’s paper on an early childhood nutrition intervention in Guatemala, and Baird, et. al’s paper on the long term effects of deworming in Kenya. All of these papers help us understand the significant effects of human capital investment on earnings – a critical issue for development policy. Heck, Baird et. al. even cite Arthur Lewis.
All of these papers use interventions from childhood and map these effects into adulthood, thereby giving a big push to focusing on early-in-life interventions. This kind of work would also be key in understanding interventions which might take a long time to manifest (as Michael pointed out) and those which address a cumulative problem. For example, a couple of weeks ago I was talking with a doctor in rural Kenya who claimed that a lot of the older women he was seeing were dying from long-term exposure to smoke from their cooking fires. He noted that this was indeed a problem for children, but there was a second bump for older women. If we wanted to see the effect of improved cookstoves on (older) adult mortality from pulmonary problems, we would need an evaluation with a significantly longer term horizon than usual.
So why don’t we see more of this kind of work?
The most obvious reason is that it takes a long time. Tenure clocks don’t run that long, and grants definitely don’t run that long. One way around this is to go look for an intervention in the past which let’s you identify an effect as Esther’s paper on Indonesia did (and from a policy perspective, please look for a program intervention – a quick look at the literature out there in this vein shows far more on negative shocks – war, famines – than on program effects). Another way around this is to look at more short term effects/variables, for example schooling in the case of deworming.
Of course, the effects might not be linear over time. Michael’s point about monitoring is key here, but we should go further and consider multiple follow up surveys – perhaps with quick qualitative work or light survey rounds to tell us when things are getting more interesting and thus that it’s time for a more heavy survey round. The usefulness of this approach was brought home to me by a recent presentation by Sebastian Galiani where he was talking about watching formalized property rights revert back (at least partially) to informality because of the transaction costs of transferring ownership.
And then you have the problem of the treated control group. If short term results show a significant benefit, and funds are available, there is a good ethical argument for providing the program to the control group as well. This is analogous to the “stopping rule” in clinical trials (although in economics we do not approach this with any of the organization including pre-program discussion that they do). This means that 10 years later you have early beneficiaries and later beneficiaries. All is not lost – fruitful estimation can be done with these two groups, as is done in the Baird, et. al. paper – and maybe we should be doing more of this. Moreover, if the treatment applies say to a certain age group, one can also get traction by using cohorts as Esther has done.
Then there’s the problem of finding the original sample. If you’ve been doing some interim work you have a better shot, but for longer follow-ups attrition can be high – for example in the Hoddinott, et. al. study, they showed up 25 years later and faced 40 percent attrition. Here’s one sin I have committed on more than one survey – not setting up pretty detailed tracking information in the baseline. Asking questions like who else in the village knows your family, getting alternate phone numbers including phone numbers of friends and families, getting a GPS reading – these are all things that can be used in a longer term follow up. The problem is, of course, that you’re not planning on the 15-20 year follow up now. Think of it as a gift for your children.
So these are some of the reasons. Other thoughts? Other ideas on what we might do?
Join the Conversation