This post, by Morten Jerven, is a contribution to an online symposium on the changing nature of knowledge production in fragile states. Be sure to read other entries by Deval Desai and Rebecca Tapscott, Lisa Denney and Pilar Domingo, and Michael Woolcock.
In 2010 I was doing research for Poor Numbers: How we are misled by African development statistics and what to do about it. I was In Lusaka, Zambia on a Wednesday afternoon, and was having a free and frank conversation with a specialist working for the UK’s Department for International Development (DfID) office there as part of the ethnographic component of my book. One of the themes we kept returning to was the problem that donors demanded evidence that was not necessarily relevant for Zambian policy makers. We were also discussing how results-oriented MDG reporting had created real outside pressure to distort statistics, with donors having the final say on what gets measured, when and how. Indeed, whenever I asked anyone in Zambia—and elsewhere in sub-Saharan Africa—“what do we know about economic growth?,” a recurring issue was how resources were diverted from domestic economic statistics to MDG-relevant statistics.
Two days later I was sitting in the Central Statistical Office in Lusaka, talking to the then only remaining member of the economic statistics division. In 2007, this division was manned by three statisticians, but when I returned in 2010, there was only one person there. The other two had been pulled from economic statistics to social and demographic statistics where there was more donor money for per diem payments. The phone rang. DfID Lusaka was on the other end. They had a problem. They had financed a report on social statistics, but the office statistician tasked with completing the report had recently travelled to Japan to participate in a generously funded training course, leaving the report incomplete. Could someone help out? And so it was that the last remaining economic statistician for the Zambian government temporarily left the office to come to the rescue.
Nicholas Waddell, a DFID Governance Adviser working on ‘Governance for Economic Development’ (G4ED) explores the links between governance and economic growth.
Should I play it safe and join a governance team or risk being a lone voice in a sea of economists and private sector staff? This was my dilemma as a DFID Governance Adviser returning to the UK after a stint in East Africa. I gambled and joined the growth specialists in DFID’s newly created Economic Development arm. A year in, I now think differently about the relationship between growth and governance.
Eradicating poverty will not be possible without high and sustained growth that generates productive jobs and brings benefits across society. Historically, this has included boosting productivity within existing sectors as well as rebalancing economies towards more productive sectors (e.g. from agriculture to manufacturing). Such structural change or economic transformation has lifted millions from poverty.
Economic transformation can have a strong disruptive effect on political governance – giving rise, for example, to interest groups that push for accountable leaders and effective institutions. As countries get richer, more effective institutions also become more affordable. Over time, economic transformation can therefore advance core governance objectives.
But this is easier said than done. Economic development is an inherently political process that challenges vested interests. Often the surest ways for elites to hold onto power and profit aren’t in step with measures to spur investment, create jobs and foster growth. Shrewd power politics can be bad economics.
How useful is 'focusing on results' for development work? It may make an organization more cost-efficient but not necessarily more effective as it is usually unrealistic, time-consuming and misleading.
How do donors aim for “results” without setting up a counterbureaucracy that disrupts rather than encourages good development programs?
A recent Independent Commission for Aid Impact report has taken the U.K. Department for International Development to task for doing just that, which in turn demands a serious reconsideration of how DfID thinks about results and accountability.
Of course, these critiques are hardly new. However this isn’t another nongovernmental organization or academic report slating the “results agenda,” but an independent body that has specifically been set up to ensure the effectiveness of aid and — based on 44 previous reports — is providing evidence about how the results agenda unfolds in practice.
In a nutshell, ICAI argues that DfID today knows better than ever before when and where taxpayers’ money is being spent, but not what that spending actually achieves. ICAI found that the results agenda has tended to prioritize short-term economy and efficiency over long-term, sustainable impact. It has brought “greater discipline” and “greater accountability for the delivery of aid” but also a focus on quantity of results over quality.
Not everything ICAI has to say is bad news; but most of it is. The ICAI findings undoubtedly hold broader relevance for other donors who are taking a similar approach to their result agenda.
What is the 'results agenda' and how does it relate to transformational change within development? The recent publication of a report from The Independent Commission for Aid Impact (ICAI), which scrutinizes UK aid spending, has brought these questions to life once again. Here are some takeaways on the report and the need for systems thinking, accountability, and flexibility from Suvojit Chattopadhyay.
Craig Valters’ Devex post, based on yet another newsworthy ICAI report, seems to have somewhat revived the debate over the ‘results agenda'. The criticism is sharper, castigating DFID for the “unintended effect of focusing attention on quantity of results over their quality” – but also one that clearly implies that the ‘results agenda’ is not well-understood or widely shared within donors like DFID. Focusing on ‘results’ cannot mean a divorce from long-term outcomes. What ICAI describes sounds more like an outputs agenda that is transactional (what your money can buy) rather than transformative (the good change).
The consequence of this bean-counting is that complex problems risk being ignored: donors and the partners they fund will tend to focus on projects, rather than systems. Also, genuine accountability along the aid-chain takes a hit due to a general break-down of trust between the different actors. So what can we do about this?
DFID really is an extraordinary institution. I spent Monday and Tuesday at the annual get together one of its professional cadres – about 200 advisers on governance and conflict. They were bombarded with powerpoints from outside speakers (including me), but still found time for plenty of ‘social loafing’, aka networking with their mates. Some impressions:
They are hugely bright and committed, wrestling to get stuff done in some of the most difficult places on the planet, familiar with all the dilemmas of ‘doing aid’ in complex environments that I talk about endlessly on this blog. A visitor muttered about the quality and nuance of discussion compared to the uncritical can-do hubris of much of what they hear in Washington.
In fact, since the Australians and Canadians wound up their development departments, DFID looks pretty well unique in the international scene – heroic keeper of the flame or aid’s Lonesome George heading for species extinction? We’ll find out over the next few years.
Aid donors are often maligned for bureaucratic procedures, a focus on short-term results at the expense of longer-term, riskier institutional change, and a technical, managerial approach to aid with insufficient focus on context, power and politics. Are these institutional barriers insurmountable? Can aid agencies create an enabling environment to think and work politically?
Tom Wingfield (top) and Pete Vowles (bottom) from DFID’s new ‘Better Delivery Taskforce’ have been trying to do just that. Here’s where they’ve got to.
For the past year DFID has been focussing on these issues and how we can both guard taxpayer’s money and have transformational impact in the countries where we work. The result has been the introduction of a comprehensive set of reforms targeting our process, capability and culture. This is about creating the conditions that allow us to better address the underlying causes of poverty and conflict, and respond effectively to the post-2015 agenda. At the heart of the reform is a revamp of DFID’s operating framework (ie the rules and principles which govern our work). Known as the ‘Smart Rules’, it can be downloaded here.
Like any institutional reform, this is a long term change process. The next 12 months provide a real opportunity to strengthen our partnerships with a wide range of partners and enhance our collective effectiveness.
These are some of the views and reports relevant to our readers that caught our attention this week.
“By the end of 2013, there will be more mobile devices on Earth than people, a new report suggests.
According to Cisco's Visual Networking Index Global Mobile Data Traffic Forecast Update, consumers' mobile appetite has grown a lot in the past year, and it shows no signs of slowing. In fact, Cisco predicts global mobile data traffic will increase 13-fold by 2017, with more than 10 billion mobile-connected devices by then. It also believes mobile network speeds will grow by seven times what it is now.” READ MORE
More DFIDistas on the blog: this time Nick York, DFID’s top evaluator and Caroline Hoy, who covers NGO evaluation, comment on Oxfam’s publication of a set of 26 warts-and-all programme effectiveness reviews.
Having seen Karl Hughes’s 3ie working paper on process tracing and talked to the team in Oxfam about evaluation approaches, Caroline Hoy (our lead on evaluation for NGOs) and I have been reading with considerable interest the set of papers that Jennie Richmond has shared with us on ‘Tackling the evaluation challenge – how do we know we are effective?’.
From DFID’s perspective, and now 2 years into the challenges of ‘embedding evaluation’ in a serious way into our own work, we know how difficult it often is to find reliable methods to identify what works and measure impact for complex development interventions. Although it is relatively well understood how to apply standard techniques in some areas – such as health, social protection, water and sanitation and microfinance – there are whole swathes of development where we need to be quite innovative and creative in finding approaches to evaluation that can deal with the complexity of the issues and the nature of the programmes. Many of these areas are where NGOs such as Oxfam do their best work.
Got dragged into DFID this week for yet another session on theories of change. This one was organized by the DFID-funded Research for Development (R4D) project (sorry, ‘portal’). A lot of my previous comments on such sessions apply – in DFID the theories of change agenda seems rather dominated by evaluation and planning (‘logframes on steroids’), whereas in Oxfam, it is mainly used to sharpen our work in programmes and campaigns. But the conversation that jumped out at me was around ‘how do we influence the researchers that we fund to use theories of change (ToCs) to improve the impact of their research?’
It’s risky to generalize about ‘academics’, but I'm going to do it anyway. Let’s apply some ToCs thinking to academia as a target. Applying ToCs to try and understand why academics don't use ToCs may feel a bit weird (like the bit in Being John Malkovich where Malkovich enters his own brain), but bear with me.