Syndicate content

Theories of Change

Does “Rational Ignorance” make working on transparency and accountability a waste of time?

Duncan Green's picture

Guest post from Paul O’Brien, Vice President for Policy and Campaigns, Oxfam America (gosh, they do have august sounding job titles, don’t they?)

As the poorest half of the planet sees that just 62 people have more wealth than all of them, collective frustration at extreme inequality is increasing.  To rebalance power and wealth, many in our community are turning to transparency, accountability, participation and inclusion.  Interrogate that “development consensus,” however, and opinions are fractured over the benefits and costs of transferring power from the haves to the have-nots.

Social Media Information OverloadIn truth, our theories of change often diverge.  Most development organizations may agree on the need to advocate for more Investment, Innovation, Information, strong Institutions and Incentives, but some organizations are genuinely committed to only one of those “I’s”, and that can be problematic:  Oxfam often finds itself choosing and moving between the relentless positivity of politically benign theories of change (e.g. we just need more “investment” or “innovation”), the moderation of those who focus exclusively on transparent “information” with no clear pathway to ensure its political relevance, and the relentless negativity of activists that think the only way to transform “institutions” or realign the “incentives” of elites is to beat them up in public.

Oxfam’s challenge is to be both explicit in our theory of change and show sophistication and dexterity in working across that spectrum.  If Oxfam’s theory of change is based on a citizen-centered approach to tackling global systemic challenges like extreme inequality, then our opportunity may be engaging the “rational ignorance” of citizens and consumers.
 

Blog post of the month: If using ‘Theories of Change’ cannot transform the way you operate, why bother?

Suvojit Chattopadhyay's picture

Each month People, Spaces, Deliberation shares the blog post that generated the most interest and discussion. In November 2015, the featured blog post is "If using ‘Theories of Change’ cannot transform the way you operate, why bother? " by Suvojit Chattopadhyay .

Learning computer skillsIn a new (and commendably short) paper, Craig Valters advocates ‘modest radicalism’ in the use of Theories of Change (ToC) as an approach to improving reflection and learning in the development sector. In this paper, Craig reflects on the role of the ToC in the context of the ‘results agenda’ and suggests four principles that could help development organisations develop knowledge and improve practice: Focus on processes; Prioritise learning; Be locally-led; and ‘Think compass, not map’. Do read the full paper!

In this post, I share some additional thoughts on the use of ToCs and how they might be improved. I start with two problems in the way we do things.

  • In development, failures are hard to detect: Often, organisations that fail find ways to mask failure – by either refusing to acknowledge failure, finding external factors, or moving on to a different desk officer/donor/location. So within the aid industry, we have a peculiar situation where it is real hard to fail – or at least, it is hard to know when a project has failed.
  • It’s harder still to ensure that projects that fail face significant consequences of failure: Organisations that implemented the failed projects should be required to make significant changes to key aspects of design or management.

Reflecting on being radical: Integrating theories of change as practice

Heather Lanthorn's picture

Ms. Gurugalpola, teaches parents and children about dental hygieneRecently, Craig Valters published new work on theories of change. He calls not for a new tool (product) but for a more careful approach (process) to practicing and engaging in development. That is, changing the state of the world for someone. And learning from it. And, ideally, communicating that learning. (Craig is pessimistic that we are near actually ushering in a learning agenda to replace the results agenda. On this, I hope he is wrong.)
 
In this post, I echo and expand on the idea of theories of change as allowing “space for critical reflection” (p. 4) and push back slightly on two of the outlined ‘key principles’ of a theory of change approach: being ‘locally led’ and thinking ‘compass, not map.’ I also include some of the comments Craig made on the original version of this post, here.

I have two disclaimers, given points raised both in the paper and in Suvojit’s follow-up blog. The first is a musing, though I have adopted the theory of change language along with the herd. I wish we could still revise it to hypotheses of change or ideas of change or stuff that might matter because we thought hard about it, looked at what had been done before, and talked to people about what could be done now (or something else catchier but far more tentative, humble, and open to updating than 'theory'). Alas. On the brighter side, Craig notes that, at least, “theory implies we have to think really hard about it, even if what we end up with is not a theory in the social science sense of the word.”
 
The second is a confession. I really like boxes and arrowsnot as the definitive product associated with a theory of change but as some means of organizing ideas that people can stand around, look at, point to, and ask, “have we learned anything about how this arrow really works?” While I don’t want to foist the need for a visual on anyone, especially if it is just going to end as a bad flowchart, I feel I should at least lightly advocate that a visual can be a useful tool for learning and may be friendlier to revisit than a lengthy narrative, and it's usually prettier. In his follow-up responses, Craig echoes that a diagram, no matter how pretty, “is not in itself a Theory of Change.” I concur.
 

3 big problems with how we think about results and development

Craig Valters's picture

How useful is 'focusing on results' for development work? It may make an organization more cost-efficient but not necessarily more effective as it is usually unrealistic, time-consuming and misleading.  

Justine Greening sees the building of Kerry Town Ebola treatment centre in Sierra LeoneHow do donors aim for “results” without setting up a counterbureaucracy that disrupts rather than encourages good development programs?

A recent Independent Commission for Aid Impact report has taken the U.K. Department for International Development to task for doing just that, which in turn demands a serious reconsideration of how DfID thinks about results and accountability.

Of course, these critiques are hardly new. However this isn’t another nongovernmental organization or academic report slating the “results agenda,” but an independent body that has specifically been set up to ensure the effectiveness of aid and — based on 44 previous reports — is providing evidence about how the results agenda unfolds in practice.

In a nutshell, ICAI argues that DfID today knows better than ever before when and where taxpayers’ money is being spent, but not what that spending actually achieves. ICAI found that the results agenda has tended to prioritize short-term economy and efficiency over long-term, sustainable impact. It has brought “greater discipline” and “greater accountability for the delivery of aid” but also a focus on quantity of results over quality.

Not everything ICAI has to say is bad news; but most of it is. The ICAI findings undoubtedly hold broader relevance for other donors who are taking a similar approach to their result agenda.
 

What Happens when 20 Middle East Decision Makers Discuss Theories of Change?

Duncan Green's picture
My first job after returning from holiday (disaster tourism in Northern Ireland – don’t ask) was to speak on Theories of Change to a really interesting group – ‘building a rule of law leadership network in the Middle East’, funded by the UK Foreign Office. The John Smith Trust has about 20 lawyers, civil servants, policemen, UN personnel and business people for a 3 week training programme. Equal numbers of men and women, from Bahrain, Iraq, Jordan, Lebanon, Oman. Chatham House rules so that’s your lot viz info.

Over the course of a year, each Leadership Fellow develops an Action Plan for reform back home, ranging from girls’ education to police training to civil society strengthening, and will work on it during their UK visit, where they get inputs from people like me, discussions and visits to the UK Parliament and elsewhere.

I was presenting on theories of change (ToCs) – here’s my powerpoint. My co-presenter (from a UK thinktank) defined a ToC as ‘a conceptual map of how activities lead to outcomes’. As you might imagine, I disagreed with the implied linearity of that. But the disagreement, and the views of those present was interesting.

Theories of Change, Stakeholders, Imagined Beneficiaries, & Stealing from Product Design. That is, Meet ‘Mary.’

Heather Lanthorn's picture

I have been thinking a lot about ‘theories of change’ this week (as I was here).  Actually, I have been thinking more about ‘conceptual models,’ which was the term by which I was first introduced to the general idea* and the term I still prefer because it implies more uncertainty and greater scope for tinkering than does ‘theory.’ (I accept that ‘theory of change’ has been branded and that I have to live with it, but I don’t have to like it.)

Regardless of the term, the approach of thinking seriously about how behavioral, social and economic change happens is important but often overlooked during the planning stages of projects/programs/policies and linked evaluations. Moreover, they are glossed over in the analysis and reporting stages, left to academic speculation in the discussion section of an evaluation paper and not informed by talking systematically to those people who were intended to benefit from the program.

I think there is growing recognition that building a theory of change is something that should happen, at least in part, backwards (among other places where this is discussed is in ‘evidence-based policy’ with the idea of a ‘pre-mortem‘ and ‘thinking step-by-step and thinking backwards‘).  That is, you start with the end goal, usually some variant of ‘peace,’ ‘satisfaction,’ ‘wellbeing,’ ‘capabilities,’** etc., in mind and work backwards as to how you are going to get there from here.

The Best Evidence Yet on How Theories of Change are Being Used in Aid and Development Work

Duncan Green's picture

If you are interested in Theories of Change (ToCs), you have to read Craig Valters’ new paper ‘Theories of Change in International Development: Communication, Learning or Accountability’ or at least, his accompanying blog. The paper draws on the fascinating collaboration between the LSE and The Asia Foundation, in which TAF gave LSE researchers access to its country programmes and asked them to study their use of ToCs. That means Craig has been able to observe their use (and abuse) in practice.

What this paper helps answer is the question I raised a while ago – will ToCs go the way of the logframe, starting out as a good idea, but being steadily dumbed down into a counterproductive tickbox exercise by the procedural demands of the aid business?

Should You Keep Innovating as a Programme Matures? Dilemmas from (another) Ground-Breaking Accountability Programme in Tanzania

Duncan Green's picture

Certain countries seem to produce more than their share of great programmes. Vietnam is one, and Tanzania appears to be another. After the much-blogged-on Twaweza workshop in Tanzania last week, I headed up North to visit the Chukua Hatua accountability programme. It’s one of my favourites among Oxfam’s governance work, not least because it has a really top notch theory of change (keep clicking) I often get asked for a good real life practical example of a ToC – in governance work, this is the best I’ve seen.

Over a series of conversations with Oxfam staff and partners, village activists, officials and others, one intriguing issue struck me: even if you start out as innovative, what happens next?

Let me explain. Chukua Hatua started out with a really interesting theory of change – adopt an evolutionary approach of variation-selection-amplification. That meant trying out lots of things in phase 1 (2010/11), then sifting through the results to identify the most successful variant(s) and scaling that up.

The variant that stood out was that of animation: training farmers selected by their communities to become animators – entrepreneurial, networked activists identifying problems in their communities and bringing people together (both villagers and those in power) to find solutions. This has worked brilliantly, so phase 2 (2012/13) has scaled that up.

Last Word to Twaweza: Varja Lipovsek and Rakesh Rajani on How to Keep the Ambition and Complexity, Be Less Fuzzy and Get More Traction

Duncan Green's picture

Twaweza’s Varja Lipovsek, (Learning, Monitoring & Evaluation Manager) and Rakesh Rajani (Head), respond to this week’s
series of posts on their organization’s big rethink.

That Duncan Green dedicated three posts on Twaweza’s ‘strategic pivot’ may signal that our work and theory of change are in real trouble, but we prefer to take it as a sign that these issues are of interest to many people working on transparency, accountability and citizen-driven change. His posts follow a terrific two day evaluation meeting. Here are a few clarifications and takeaways.

Spiritual matters first. We very much believe that Twaweza’s soul remains intact: we want to contribute towards change in complex systems in East Africa, by promoting and enabling citizens to be active agents and shape their lives. Our experience over the past four years has made us question much of how we ‘do’ citizen agency, but we are not quite throwing out the baby with the bathwater.

For example, in our original approach we didn’t want to be prescriptive about citizen action; we wanted to expand choices and leave it up to people to decide, what we called an ‘open architecture’ approach to social change. Sounds good; problem is that it doesn’t work so well in practice and the evidence of successful change suggests a need for less openness and more focus. New evidence about the bandwidth that poor people have to make good decisions provides useful insights on what one can realistically expect people to do.

The War for Twaweza's Soul: The Hunger for Clarity and Certainty v the Demands of Complexity

Duncan Green's picture

This is the last in a series of three posts on Twaweza, a fascinating NGO doing some pioneering work on accountability in East Africa, whose big navel gaze I attended last week. Post one covered Twaweza’s theory of change and initial evaluation results; yesterday I got onto the critique of its thinking and action to date. Today I’m digging deeper into some of the underlying issues.

Given its rethink, Twaweza is now contemplating a shift in direction – while keeping its focus on citizen agency, focus in on education (rather than try and cover education, health and water); reduce the number of partners; do more things on its own (eg research or education programming); expand successful areas such as policy and advocacy; do more experiments to uncover what works and help the organization ‘fail faster’ and so move on to new stuff.

Plenty of good ideas in there, but it also seems to me to mark an intellectual retreat from the initial commitment to finding new ways to achieve change in complex systems. I think there’s a strong case for digging deeper into complexity, rather than retreating from it. One suggestion that moves in the right direction is to set up a ‘positive deviance lab’, dedicated to detecting and then understanding examples of success in citizens’ action across East Africa.


Pages