Syndicate content

Theory of Change

Thinking through funnels of attrition

Heather Lanthorn's picture

When first introduced to the idea of a funnel of attrition (my early attempt at a slightly more nuanced and symmetric — but still generic — version is here), I largely thought of it as a useful heuristic for thinking about sample size calculations, by being forced to think about issues of awareness and take-up as well as a few steps along a causal chain between initial participation or use and longer terms outcomes of interest.

More recently (including here), I  have tried to use it as a tool for thinking about articulating assumptions in a theory of change about where people might ‘fall out of’ (or never join) an intervention, thus leaving the funnelMore specifically, I tried (along with colleagues) using it as a goal for a conversation with implementing partners (that is, “let’s map out the funnel of attrition”), tackling the question from multiple perspectives. Various perspectives were brought in using personae, which I created beforehand relying partially on average results from the baseline as well as some stylizing to try to bring certain features into the conversation. At first I feared being overstylized but, in the end, I think I had too little detail. I reviewed my notes from The Inmates are Running the Asylum and was reminded of the importance of specificity, even at the expense of accuracy.

I liked this idea for guiding a conversation because the funnel of attrition is a little more straightforward than a full theory of change but, in constructing it, you still end up articulating some central assumptions, which can be added to thinking about change may/not happen. It seems like a handy building block in a well-considered theory of change.

Found a positive impact, published in a peer-reviewed journal. What more do we need?

Urmy Shukla's picture

Family utilizes protective malaria bed nets in their home, Nigeria In this blog, we advocate the importance of in-depth reporting on implementation processes, evaluation processes, and relevant contextual details of interventions and linked evaluations. This will facilitate research transparency, as well as assessments of both learning and the potential for generalizability beyond the original study setting (learning lessons from ‘there’ for ‘here,’ but not necessarily promoting the strict and exact duplication of a program from one setting to another, in line with an understanding of external validity that is appropriate for the social sciences in development).
 
We start with a hypothetical scenario of an intervention and associated evaluation, based on too-frequent experiences in the impact evaluation space. We hope that it doesn’t sound familiar to those of you who have been involved in evaluation or have tried to make sense of evaluation results -- but suspect that it will.
 
A research team, connected to a larger research and evaluation organization, ran a study on an intervention. For reasons of statistical and political significance, they have deemed it sufficiently successful and worthy of scaling up, at least in a very specific new setting. 
 
The intervention sought to overcome the following problem, for which there are supply-side and demand-side issues. People in malarious areas may procure a bednet (whether for free or for a positive price), but they do not always follow-through with maintenance (re-treatment or replacement).
 
For supply, the private sector only sporadically offers retreatment and replacement, and it is expensive, while the public sector does not always have supplies available. The intervention, therefore, concentrates provision of this service at a specific time and place through temporary service centers.
 
For demand, people with nets often don’t understand the need for retreatment and, even if they do, continuously put off doing so. The intervention, therefore, included a non-monetary incentive for which there is local demand (in this case, soap) to be picked up at the time of net retreatment.

Weekly wire: The global forum

Roxanne Bauer's picture
World of NewsThese are some of the views and reports relevant to our readers that caught our attention this week.
 

Do international NGOs still have the right to exist?
The Guardian
It’s highly unlikely that corporate bosses regularly ask themselves if their businesses have a right to exist. Their goal is to sell stuff and make a profit. But if your goal is to alleviate poverty and human suffering – in the face of statistics showing mixed outcomes – is this, in fact, the most important question an International NGO can ask of themselves? At the BOND conference last week, in a session entitled How can INGOs survive the future, Penny Lawrence, the deputy CEO of Oxfam stated bluntly: “we need to earn the right to survive the future.” It is like the sector’s very own Damascene moment.

Changing views of how to change the world
Brookings, Future Development blog
World leaders concluded three large agreements last year. Each represents a vision of how to change the world. The Addis Ababa Action Agenda on financing for development agreed to move from “billions to trillions” of cross-border flows to developing countries. The agreement on universal sustainable development goals (SDGs) sets out priorities (albeit a long list) for what needs to change. The Paris Agreement on climate change endorses a shift to low-carbon (and ultimately zero carbon) economic growth trajectories. There is a common thread to these agreements. They each reflect a new theory of how to change the world that is not made explicit but has evolved as a matter of practice. Understanding this new theory is crucial to successful implementation strategies of the three agreements.
 

Four ways open data is changing the world

Stefaan Verhulst's picture

Library at Mohammed V University at Agdal, RabatDespite global commitments to and increasing enthusiasm for open data, little is actually known about its use and impact. What kinds of social and economic transformation has open data brought about, and what is its future potential? How—and under what circumstances—has it been most effective? How have open data practitioners mitigated risks and maximized social good?

Even as proponents of open data extol its virtues, the field continues to suffer from a paucity of empirical evidence. This limits our understanding of open data and its impact.

Over the last few months, The GovLab (@thegovlab), in collaboration with Omidyar Network (@OmidyarNetwork), has worked to address these shortcomings by developing 19 detailed open data case studies from around the world. The case studies have been selected for their sectoral and geographic representativeness. They are built in part from secondary sources (“desk research”), and also from more than 60 first-hand interviews with important players and key stakeholders. In a related collaboration with Omidyar Network, Becky Hogge (@barefoot_techie), an independent researcher, has developed an additional six open data case studies, all focused on the United Kingdom.  Together, these case studies, seek to provide a more nuanced understanding of the various processes and factors underlying the demand, supply, release, use and impact of open data.

After receiving and integrating comments from dozens of peer reviewers through a unique open process, we are delighted to share an initial batch of 10 case studies, as well three of Hogge’s UK-based stories. These are being made available at a new custom-built repository, Open Data’s Impact, that will eventually house all the case studies, key findings across the studies, and additional resources related to the impact of open data. All this information will be stored in machine-readable HTML and PDF format, and will be searchable by area of impact, sector and region.

Four principles for Theories of Change in global development

Craig Valters's picture

The stratospheric rise of the Theory of Change approach continues. In a new paper published on September 15, 2015, I argue that taking a Theory of Change approach demands a radical shift towards more and better learning in development thinking and practice.

Local farmers attend a workshop on ecology and social organization in Vila Da Canoas, in the Amazon region of Brazil near ManausHang on, do we know what a Theory of Change approach actually is?

At a workshop at ODI in April 2015, we sought to work out how different people were using the term, for what purpose, and with what effects. More detail on that can be found here and here. What’s emerged is that the term ‘Theory of Change’ is being used in at least three overlapping ways:

As a discourse, asking ‘what’s your Theory of Change?’ has become an increasingly fashionable way interrogate someone’s assumptions about change (and flummox newcomers to the terminology).

As a tool, it’s rapidly rivalling (and being used in conjunction with) the log frame. Here it’s often used as a way of making explicit the assumptions connecting (watch out, here comes aid jargon) activities, outputs and outcomes in reporting for donors.

Taking a Theory of Change approach will likely include use of a tool in some form, but is broader, reflecting a desire to embed a critical and adaptive approach in organisational practice. This is perhaps the most exciting, as it builds in what we know about how aid organisations can make effective contributions to social change in complex environments.

So where do we go from here?

The following principles (not rules) seek to ground Theory of Change approaches in this emerging knowledge – and are rooted in a concern with persistently damaging problems within the industry.

However, the aim is to not to be prescriptive: debate them, critique them, and develop your own!
 

Where have we got to on Theories of Change? Passing fad or paradigm shift?

Duncan Green's picture

Gum Arabic farmers at Hilat Ismaiel, North Kordofan, SudanTheories of change (ToCs) – will the idea stick around and shape future thinking on development, or slide back into the bubbling morass of aid jargon, forgotten and unlamented? Last week some leading ToC-istas at ODI, LSE and The Asia Foundation and a bunch of other organisations spent a whole day taking stock, and the discussion highlighted strengths, weaknesses and some looming decisions.

(Summary, agenda + presentations here)

According to an excellent 2011 overview by Comic Relief, ToCs are an "on-going process of reflection to explore change and how it happens – and what that means for the part we play". They locate a programme or project within a wider analysis of how change comes about, draw on external learning about development, articulate our understanding of change and acknowledge wider systems and actors that influence change.

But the concept remains very fuzzy, partly because (according to a useful survey by Isobel Vogel) ToCs originated from two very different kinds of thought: evaluation (trying to clarify the links between inputs and outcomes) and social action, especially participatory and consciously reflexive approaches.

At the risk of gross generalization, the first group tends to treat ToCs as ‘logframes on steroids’, a useful tool to develop more complete and accurate chains of cause and effect. The second group tend to see the world in terms of complex adaptive systems, and believe the more linear approaches (if we do X then we will achieve Y) are a wild goose chase. These groups (well, actually they’re more of a spectrum) co-exist within organisations, and even between different individuals in country offices.

Blog Post of the Month: The Best Evidence Yet on How Theories of Change are Being Used in Aid and Development Work

Duncan Green's picture
Each month, People, Spaces, Deliberation shares the blog post that generated the most interest and discussion.

In September 2014, the most popular blog post was "The Best Evidence Yet on How Theories of Change are Being Used in Aid and Development Work"

In this post, Duncan Green, provides an overview of Craig Valters’ new paper ‘Theories of Change in International Development: Communication, Learning or Accountability’  The paper, and Duncan's blog post, help answer the question: will Theories of Change "go the way of the logframe, starting out as a good idea, but being steadily dumbed down into a counterproductive tickbox exercise by the procedural demands of the aid business?"

Read the blog post to learn more!
 

What Use is a Theory of Change? 6 Benefits, and Some Things to Avoid.

Duncan Green's picture

Whether in the back of a 4×4 in Tanzania, or in seminar rooms in Oxfam house, I seem to spend an increasing amount of my time discussing theories of change. Oxfamers seem both intrigued and puzzled – what are they? What are they for? The answers aren’t simple and, as social scientists like to say, they are contested. But here’s what I currently think.

What is a theory of change? A way of working and thinking, and a set of questions. Aerobics for the imagination – not a form to fill in (and most definitely not logframes on steroids). Nor is it a typology or (a personal bête noire) an insanely complicated diagram that no-one coming after you can understand (see example, right). More here.

How does (or should) a good theory of change improve our work (or ‘add value’ as the marketing wannabes insist on saying)?

What is a Theory of Change and How Do We Use It?

Duncan Green's picture

I’m planning to write a paper on this, but thought I’d kick off with a blog and pick your brains for references, suggestions etc. Everyone these days (funders, bosses etc) seems to be demanding a Theory of Change (ToC), although when challenged, many have only the haziest notion of what they mean by it. It’s a great opportunity, but also a risk, if ToCs become so debased that they are no more than logframes on steroids. So in internal conversations, blogs etc I’m gradually fleshing out a description of a ToC. When I ran this past some practical evaluation Oxfamers, they helpfully added a reality check – how to have a ToC conversation with an already existing programme, rather than a blank sheet of paper?

But first the blank sheet of paper. If you’re a regular visitor to this blog, you’ll probably recognize some of this, because it builds on the kinds of questions I ask when trying to understand past change episodes, but throws them forward. Once you’ve decided roughly what you want to work on (and that involves a whole separate piece of analysis), I reckon it’s handy to break down a ToC into four phases, captured in the diagram.