“Consider the story of Mrs. Braulia Parra, who lives with a family of seven in a poor neighborhood in Monterrey, Mexico, in a home with cardboard walls and dirt floors. Illiterate and inexperienced in the workplace, Mrs. Parra took her first $150 loan from ADMIC, a local microlender. The loan allowed her to buy yarn and other sewing supplies to make handsewn decorations. Each week she sells about one hundred handmade baskets, dolls and mirrors, going door-to-door in her neighborhood. After ten loans, Mrs. Parra had earned enough to install a toilet in her modest home, as well as an outdoor shower. Building a second floor was next in her sights.”
These stories inspire us, but we have to be careful. Stories are dangerous.
Stories are dangerous because they can be wrong. One program I am familiar with provided cash transfers to poor households. In the pilot, the government rolled out the program to a set of randomly selected treatment villages and compared them to a group of control villages. The program had myriad positive impacts (health, education, and much more!), but one area where the program did not have a clear impact was in housing materials. Yet when the researchers visited beneficiary villages and spoke to participants, they highlighted how they had improved their houses with the resources. The data showed that housing did indeed improve in participating villages: It just didn’t improve any more there than in comparison villages. In other words, the program was not responsible for housing improvements. But people received resources and they improved their houses, so their story (when speaking to the donors, at least) was that these resources had led to home improvements. If my goal were to start a program to improve housing, and I focused on the stories I heard, I might say: This is a great way to improve housing! I would be mistaken. Rather, it’s a great way to let households invest in whatever they most need, which turns out to be insurance and livestock rather than housing.
In the United States, moving anecdotes in which parents observe their children develop the symptoms of autism soon after vaccination led many people not to vaccinate their children long after the vaccination-autism link had been disproved. In both of these examples, through no ill will of the storytellers, there is potential for grave policy errors.
Stories are dangerous because they can be too simple. The economist Tyler Cowen gave a great talk (video; transcript) on stories where he explained that “narratives tend to be too simple. The point of a narrative is to strip it way … Most narratives you could present in a sentence or two.”
Here are three of these kinds of stories: We lent this woman 20¢ to start her own bamboo stool business and she lifted herself out of poverty. We gave this child’s family $3 per month and they were able to send her to school. We provided training and women now have their own businesses and more freedom.
What do these stories miss? They miss that reality is messy and complicated. These stories miss, perhaps, that only a small fraction of beneficiaries experienced these benefits, or that the program only worked in certain contexts, or that not all handsewn decoration producers are equally effective entrepreneurs.
The inspiring microcredit story above and others like it have played a major role in motivating the microfinance movement, whereas evidence from randomized trials in India and the Philippines tell a much more complex tale, where microcredit has some positive impacts (lower spending on alcohol and tobacco in India, better ability to cope with risk in the Philippines) but also some … non-positive impacts (no increase in consumption in India, slightly lower subjective well-being in the Philippines).
The Upside. At the same time, stories can flesh out what quantitative narratives miss. In an evaluation of conditional cash transfers in Tanzania, we observed that beneficiary households were purchasing more livestock, especially chickens. Only through stories told in focus groups did we understand that these were seen as small business opportunities (selling eggs and chicks).
Take Away. The philosopher Bertrand Russell wrote that one goal of education is to enlarge people’s sympathy:
"It may only go so far as sympathy with suffering which is portrayed vividly and touchingly, as in a good novel; it may, on the other hand, go so far as to enable a man to be moved emotionally by statistics. This capacity for abstract sympathy is as rare as it is important."
Most of us aren’t quite there yet: Vivid and touching tales move us more than statistics. So let’s listen to some stories. Let’s be moved. Then let’s look at some hard data and rigorous analysis before we make any big decisions.
1) Pratham Books
2) Yuri Kozyrev (2002)/WB
Follow PublicSphereWB on Twitter