This week is the World Bank’s annual conference on development economics. One of the papers being presented is by my colleague Kate Orkin (together with co-authors Tanguy Bernard, Stefan Dercon and Alemayehu Taffesse) and takes a look at a video intervention and its impact on aspirations among poor folks in Ethiopia. In particular, what Kate and her co-authors are asking is: can we shift aspirations and behavior by showing people more of what is possible?
So this paper follows in a chain of papers that have looked at the effect of video interventions – starting with Chong and La Ferrara and Jensen and Oster’s work on the female role model effects of soap operas, and Berg and Zia’s work on financial behavior (which Bilal blogged about here). But this one takes a more macro, more overall life-goal kind of approach. Actually the best way to describe it is that it’s about aspirations. And to be precise, let’s define what those are. Bernard and co. actually spend a fair amount of time on this, but I think its best summed up as: “an aspiration expresses a preference for a ‘state of the world’ where the relevant goal is achieved, instead of other states.”
Now how do they go about understanding whether a video can shift this? They start with a relatively poor area of Ethiopia. They select a bunch of villages. And within each village, they randomly show a video to some of the folks. The video is a documentary that captures four stories (from two men and two women) who improved their socio-economic conditions despite starting from a low base. Bernard and co. note that these videos capture a couple of main themes: 1) these folks were pretty average (in terms of wealth) compared to those around them -- they just took different paths, 2) people who surround these success stories cite characteristics that matter – perseverance, determination and reliability, and 3) these folks didn’t get any outside help (If you want to take a look, a sample, with subtitles is here).
Now, on to the estimation. One question to worry about is: what’s the simple effect of showing up in a poor rural village with a video? Indeed, many of the folks in these communities don’t spend a lot of time in front of a screen – even on an annual basis. So to deal with this Bernard and co. also randomly select a bunch of folks in each village to see a placebo video – one that features Ethiopian traditional dance (which would be quite aspirational for me – I’ve tried and failed at this). And to give us some sense of whether there might be spillover effects, in some villages they give these videos to a much larger group of people (24 households versus 6).
The placebo is going to give us some hope of separating out the pure video effect from the content of the video effect. But one thing to keep in mind is that if there is a combined video and content effect this could put some pressure on their power. Another neat thing is that Bernard and co. really paid close attention to who could go to the video and who could not – tickets were labelled with names and color-coded by content. And this was checked at the door. Another thing which this evaluation shows is how to make do under budget constraints. They aren’t going to town on the intensity/spillover set-up (e.g. more variation in intensity) – opting instead for a really intense and a not intense option (the 24 vs 6 mentioned above). And, what’s more, they also don’t collect data on the additional 18 households in the intense treatment villages.
Before getting to the results, a note on the data. There is a baseline and then a follow-up right after the screening, with a second follow-up six month later. This is neat because it allows them to see if certain things decay or persist. They measure aspirations in four dimensions – income, wealth, social status, and children’s educational attainment. And they ask separately about aspirations (what people would like to achieve) and expectations (what people thought they would reach in 10 years).
So what do they find? Putting the aspirations together in an index, the documentary (relative to the control) has a significant and positive effect on aspirations right after the screening, and a significantly larger effect than the placebo video. This effect persists over time (i.e. to the six month follow-up), although it does get smaller and the gap with the placebo narrows. Looking at the individual components of the aspiration index separately, this result seems to be driven by the aspirations for their children’s education. This is an interesting twist because the documentaries don’t tackle the subject of education directly – none of the subjects talk about literacy or formal education as a factor in their success. Bernard and co. make an interesting argument throughout about the effects of a video – that it provides a broad basis on which to think about aspirations, rather than suggesting particular behaviors. Folks follow up these aspirations with concrete behaviors – six months after the video, households in the treatment group are significantly more likely to have kids enrolled in school than the control group and (at 10 percent significance) spend more money on education. However, for these variables, Bernard and co. can’t reject that the documentary and placebo videos have equal effects.
Other economic behaviors also seem to be responding to the video. In particular, Bernard and co. find that (at 10 percent significance) that documentary video watchers have higher savings and credit balances six months later. While they can’t reject equality with the placebo, the coefficients are much higher on the documentary watchers – it looks like (statistical) power is an issue here. In terms of spillovers, the data indicate that folks talked about these movies quite a bit. In terms of actual spillover impacts on behavior, the one result that comes through is that having more intense treatment in a village gets you bigger impact on education spending.
This work gives us something to chew on – not only in terms of results – but also in terms of thinking about measurement and broader methodological issues for less concrete outcomes such as aspirations. Here’s looking forward to more work in this area.
So this paper follows in a chain of papers that have looked at the effect of video interventions – starting with Chong and La Ferrara and Jensen and Oster’s work on the female role model effects of soap operas, and Berg and Zia’s work on financial behavior (which Bilal blogged about here). But this one takes a more macro, more overall life-goal kind of approach. Actually the best way to describe it is that it’s about aspirations. And to be precise, let’s define what those are. Bernard and co. actually spend a fair amount of time on this, but I think its best summed up as: “an aspiration expresses a preference for a ‘state of the world’ where the relevant goal is achieved, instead of other states.”
Now how do they go about understanding whether a video can shift this? They start with a relatively poor area of Ethiopia. They select a bunch of villages. And within each village, they randomly show a video to some of the folks. The video is a documentary that captures four stories (from two men and two women) who improved their socio-economic conditions despite starting from a low base. Bernard and co. note that these videos capture a couple of main themes: 1) these folks were pretty average (in terms of wealth) compared to those around them -- they just took different paths, 2) people who surround these success stories cite characteristics that matter – perseverance, determination and reliability, and 3) these folks didn’t get any outside help (If you want to take a look, a sample, with subtitles is here).
Now, on to the estimation. One question to worry about is: what’s the simple effect of showing up in a poor rural village with a video? Indeed, many of the folks in these communities don’t spend a lot of time in front of a screen – even on an annual basis. So to deal with this Bernard and co. also randomly select a bunch of folks in each village to see a placebo video – one that features Ethiopian traditional dance (which would be quite aspirational for me – I’ve tried and failed at this). And to give us some sense of whether there might be spillover effects, in some villages they give these videos to a much larger group of people (24 households versus 6).
The placebo is going to give us some hope of separating out the pure video effect from the content of the video effect. But one thing to keep in mind is that if there is a combined video and content effect this could put some pressure on their power. Another neat thing is that Bernard and co. really paid close attention to who could go to the video and who could not – tickets were labelled with names and color-coded by content. And this was checked at the door. Another thing which this evaluation shows is how to make do under budget constraints. They aren’t going to town on the intensity/spillover set-up (e.g. more variation in intensity) – opting instead for a really intense and a not intense option (the 24 vs 6 mentioned above). And, what’s more, they also don’t collect data on the additional 18 households in the intense treatment villages.
Before getting to the results, a note on the data. There is a baseline and then a follow-up right after the screening, with a second follow-up six month later. This is neat because it allows them to see if certain things decay or persist. They measure aspirations in four dimensions – income, wealth, social status, and children’s educational attainment. And they ask separately about aspirations (what people would like to achieve) and expectations (what people thought they would reach in 10 years).
So what do they find? Putting the aspirations together in an index, the documentary (relative to the control) has a significant and positive effect on aspirations right after the screening, and a significantly larger effect than the placebo video. This effect persists over time (i.e. to the six month follow-up), although it does get smaller and the gap with the placebo narrows. Looking at the individual components of the aspiration index separately, this result seems to be driven by the aspirations for their children’s education. This is an interesting twist because the documentaries don’t tackle the subject of education directly – none of the subjects talk about literacy or formal education as a factor in their success. Bernard and co. make an interesting argument throughout about the effects of a video – that it provides a broad basis on which to think about aspirations, rather than suggesting particular behaviors. Folks follow up these aspirations with concrete behaviors – six months after the video, households in the treatment group are significantly more likely to have kids enrolled in school than the control group and (at 10 percent significance) spend more money on education. However, for these variables, Bernard and co. can’t reject that the documentary and placebo videos have equal effects.
Other economic behaviors also seem to be responding to the video. In particular, Bernard and co. find that (at 10 percent significance) that documentary video watchers have higher savings and credit balances six months later. While they can’t reject equality with the placebo, the coefficients are much higher on the documentary watchers – it looks like (statistical) power is an issue here. In terms of spillovers, the data indicate that folks talked about these movies quite a bit. In terms of actual spillover impacts on behavior, the one result that comes through is that having more intense treatment in a village gets you bigger impact on education spending.
This work gives us something to chew on – not only in terms of results – but also in terms of thinking about measurement and broader methodological issues for less concrete outcomes such as aspirations. Here’s looking forward to more work in this area.
Join the Conversation