Anecdote + Anecdote = Anecdata?

One of the most difficult barriers in the field of communication and development is the lack of quantitative empirical evidence that demonstrates the effect of communication on development. When we argue that communication is central to development and increases development effectiveness, economists often raise an eyebrow and ask "Where's the data?" It's a legitimate question. And it's a question we don't have an answer to - yet.

Communication effects are often indirect and there may be a considerable time lag between an intervention and any tangible impact. There are some studies that show that a free press curbs corruption and thereby improves public service delivery (Reinikka and Svensson) and increases government responsiveness to crises (Sen; Besley and Burgess). In both cases the media effect on development is indirect: It works through government accountability. On the other hand we have a significant amount of case studies and anecdotes that illustrate communication effects. We hear from journalists that participate in communication training and return to their media outlets to produce stories that contribute to a better understanding of people's rights. We know of citizen uprisings where investigative journalists were instrumental in uncovering corrupt behavior and abuse of office by leading politicians. We also hear of judges in countries with strict religious doctrines that overturn draconian punishments because the public, inspired by what the media shows them of the world, stands up against them.

But again - it's all indirect, and most is anecdotal. As World Bank President Robert Zoellick likes to quote: "The plural of anecdote is not data." Or is it? Roland Rich from the United Nations Democracy Fund (UNDEF) argued in a recent talk at the Bank that we need to rethink our understanding of "evidence." He advocated for the use of the term "anecdata" - accepting examples from practice as evidence that communication works for development. He argued that it is simply impossible to produce reliable quantitative data on effects that may lag 20 years or so behind the actual intervention. Why would we assume that success stories do not demonstrate communication effects? The fact that it may take a long time for a communication measure to work does not mean that communication is ineffective. Looking at the state of the Third World and the (limited) success of development aid we really need to say: better in 20 years than never.

Having been socialized as empirical social scientist, I'm myself not quite convinced of the evidentiary value of "storytelling" - I don't need to tell you what development economists think about it. But what would an evaluation framework for communication effects on development look like? How can we assess indirect effects? What factors would communication work through? How long does it take for, lets say, a Freedom of Information Act to have an actual effect on the well-being of a society? What part of society would we have to look at to find the effect? What measures would be appropriate?

Or - should we actually utilize "anecdata" after all?