In this blog, we advocate the importance of in-depth reporting on implementation processes, evaluation processes, and relevant contextual details of interventions and linked evaluations. This will facilitate research transparency, as well as assessments of both learning and the potential for generalizability beyond the original study setting (learning lessons from ‘there’ for ‘here,’ but not necessarily promoting the strict and exact duplication of a program from one setting to another, in line with an understanding of external validity that is appropriate for the social sciences in development).
We start with a hypothetical scenario of an intervention and associated evaluation, based on too-frequent experiences in the impact evaluation space. We hope that it doesn’t sound familiar to those of you who have been involved in evaluation or have tried to make sense of evaluation results -- but suspect that it will.
A research team, connected to a larger research and evaluation organization, ran a study on an intervention. For reasons of statistical and political significance, they have deemed it sufficiently successful and worthy of scaling up, at least in a very specific new setting.
The intervention sought to overcome the following problem, for which there are supply-side and demand-side issues. People in malarious areas may procure a bednet (whether for free or for a positive price), but they do not always follow-through with maintenance (re-treatment or replacement).
For supply, the private sector only sporadically offers retreatment and replacement, and it is expensive, while the public sector does not always have supplies available. The intervention, therefore, concentrates provision of this service at a specific time and place through temporary service centers.
For demand, people with nets often don’t understand the need for retreatment and, even if they do, continuously put off doing so. The intervention, therefore, included a non-monetary incentive for which there is local demand (in this case, soap) to be picked up at the time of net retreatment.
Storytelling is essential to persuasion. But how do we decide which stories to tell? Heather Lanthorn reviews median impact narratives and explains why they are more than just window dressing.Way back in January, an interesting conversation took place on the Development Impact blog; I am just catching up on the conversation. The conversation featured a 7 January guest post by Bruce Wydick, who advocated the idea of “median impact narratives,” and subsequent commentary.
In short, Bruce recognizes that even when solid quantitative causal evidence exists,
“A good narrative soundly beats even the best data. Economists and scientists of all ilks need to digest what for many is an unpleasant fact: In the battle for hearts and minds of human beings, narrative will consistently outperform data in its ability to influence human thinking and motivate human action.”
Bruce further points out (to take a bit of liberty with his words) that, sadly, Stalin was right about at least one thing (to paraphrase): a single death is a tragedy while a million deaths is a statistic. And people do rather better with making sense of and feeling empathy for a single victim (or success) than for a large number of statistical people and their myriad outcomes (e.g.). This leads Bruce to an important question: how to choose which individuals to highlight? What Bruce’s post gets around to (and as he confirmed with me in a follow-up email, thank you!) is really a question about sampling for qualitative research and placing (and valuing) anecdotes within the context of the study sample and population.
We have mentioned it many times on this blog - CommGAP is no more. But our work lives on! Just before we closed shop at the end of October, we were able to publish three more publications directly aimed at governance practitioners that we hope you will find useful. Please check out the new facilitators guide People, Politics and Change: Building Communication Capacity for Governance Reform, the trainer's guide Generating Genuine Demand for Accountability Through Communication, and the case study compendium Changing Norms is Key to Fighting Everyday Corruption.