Syndicate content

So what exactly is the “science of delivery”?

Adam Wagstaff's picture

The World Bank’s president, Jim Kim, has now made two major speeches outlining his vision for the institution – one at the Annual Meetings the other at Georgetown University on April 2 ahead of the upcoming Spring Meetings.

Several themes are emerging. Two are easy to grasp and likely to resonate strongly with Bank staff and stakeholders: “ending poverty” and “boosting shared prosperity”. For years the Bank has seen fighting poverty as its mission. It has made major contributions in the areas of measuring and monitoring poverty – Bank staff have authored many of the world’s most-cited publications with poverty in the title. The Bank’s work at the country level has always had a strong anti-poverty focus. “Ending” poverty – rather than merely “fighting” it – is a natural next step. The idea of “boosting shared prosperity” also resonates. While economic growth is still seen as the principal driver of poverty-reduction, the goal has always been pro-poor growth – a concept that links naturally to the idea of “shared prosperity”.

One recurrent theme in President Kim’s speeches that’s less easily grasped is “the science of delivery”. Surprisingly – after all, it’s catchy and sounds like it’s been around forever – the expression “science of delivery” is actually a relatively new one. Google Scholar throws up just four publications with “science of delivery” in the title, the most-cited of which has been cited just 29 times, the second most cited has been cited just once, and the other two have never been cited. (To put this in context the 1,000 most-cited publications with “poverty” in the title have been cited collectively over 300,000 times.) The general Google search engine throws up more, but the one that appears top is the publication that’s been cited 29 times. Interestingly, the next two items both point to a video of a lecture on the “science of delivery” by Bank staffer Ani Dasgupta.

The fact that the expression “science of delivery” is so new should come as a relief to those Bank staff who think it must be old but feel nervous admitting they don’t quite know what it means. But its newness also presents them with an opportunity – to help concretize a new expression.

Solace in Molière? 

So what does the “science of delivery” mean? It’s apparently a concept that Jim Kim developed with reference to health care while he was President of Dartmouth College. Writing in 2010 for the American Council for Education, Kim wrote:

We cannot address the ever-increasing, unsustainable costs of health care without getting to the foundation of how care is provided. Nor can we achieve the social and moral goals we share – care that is safe, appropriate, effective, and high quality for every patient, in every community – without rethinking and redesigning delivery. Real improvements require a multidisciplinary approach that will bring the best minds to focus on the problem. Experts in management, systems thinking and engineering, sociology, anthropology, economics, medicine, health policy, and other fields must join together to fix the delivery system.”

The Lancet’s July 2012 portrait of Kim quotes the Bank’s President as saying:

The enormous investments that have been made in global health should have led to what we might have called a science of implementation and execution, but this has not happened. In the US and developed world health-care systems, we focus so much on discovery, on the basic science of disease, and the development of new treatments. We have just not focused on the enormous complexity of delivering health care in a way that keeps people healthy – that provides high quality health care at low cost.”

Kim’s message – generalized – is that the world has invested too much in what to deliver and too little in how to deliver it, with the result that “it” often fails to reach and benefit people.

That’s a great message. And it’s one that’s completely consistent with the Bank’s everyday work.

To take just one example, in 2004 Mariam Claeson and I wrote a report “The Millennium Development Goals for Health: Rising to the Challenges”. One of our chapters was titled “Effective Interventions Exist – They Need to Reach More People”. In it we argued:

It is not a lack of interventions that is the main obstacle to faster progress toward the Millennium Development Goals – it is the low levels of use, especially among the poor, of existing effective interventions. Globally, the use of effective interventions for child health is typically below 50 percent, and in many poor countries the figure is much lower. Using all known interventions appropriately – achieving 99 percent coverage rates – could avert 63 percent of child deaths and 74 percent of maternal deaths.”

Much of the discussion in the global health community at the time assumed that bednets, vaccines and breastfeeding could all be dropped by helicopter like manna from heaven. But that wasn’t the way our colleagues in the Bank were thinking. With their help, ideas and experiences, Mariam and I were able to assemble a rich report documenting the constraints to faster progress on the health MDGs, and how to remove them. Extra government spending was needed, but it wasn’t sufficient. Households aren’t passive – they deliver many key interventions themselves, and they demand (or don’t demand) services from others. It can’t be taken for granted that (professional) service providers will be present when they’re needed, and will deliver the required service to the best of their abilities. And it can’t be taken for granted that governments have the policies and institutions in place to make sure the necessary changes happen.

I don’t think these messages were especially novel to Bank staff in the health sector. And you see the same broad messages in Bank reports in other sectors. It’s precisely these issues that Bank staff focus on in their reports and in their day-to-work with countries. Whether it’s nutrition, secondary education or rural roads, Bank staff mostly leave the “what to deliver” question to others (including the UN technical agencies) and focus instead on the “how to deliver it” question – albeit at the level of the sector rather than at the level of the individual clinic, school and road-construction team.

Just as Molière’s bourgeois gentilhomme spent his whole life speaking prose without knowing it, World Bank staff have in effect spent their careers being a “delivery scientist” without knowing it! I’m not suggesting Bank staff all add “delivery scientist” to their business cards. But I think it is important that the “science of delivery” not be seen as something different from what Bank staff are already doing in their everyday work.

The Kim gauntlet

That’s not to say the Bank couldn’t be doing the “science of delivery” better. The Lancet’s portrait of Kim quotes him as saying:

The spread of innovation and going to scale are critical, and the great development organizations in the world have not been as effective as they could be in tackling this issue.”

I suspect all Bank staff and World Bank stakeholders would agree. The institution can doubtless do better at managing, synthesizing and disseminating global and local knowledge about how to deliver goods and services effectively. It can doubtless do better at generating and disseminating knowledge gleaned through its own operational work – not only about how to deliver goods and services, but how to deliver development assistance. It can doubtless also do better at ensuring that its operations are built around this knowledge, and are able to adapt as new knowledge emerges during implementation, building perhaps on the “Problem-Driven Iterative Adaptation” approach sketched out by Matt Andrews, Lant Pritchett, and Michael Woolcock.

So, yes, we can think of much of the everyday work of the World Bank as the “science of delivery”. But like all science, the “science of delivery” needs to keep moving forward. What Jim Kim is saying, I think, is that during his presidency he wants the World Bank to give it a sharp push forward.

Amen to that.

 

Comments

Submitted by Patricio V Marquez on
Good piece Adam. You arguments are correct as indeed the “how to deliver it” question is and should be at the core of our work. And failures have resulted in some cases when the management of projects have concentrated on the "what to deliver”question and not on the “how to deliver it” question, particularly the “how to" ADAPT not mechanically ADOPT "generic" what/how to the heterogeneous political, social and institutional realities of countries. One way to mitigate this ever present risk would be to focus, as part of the ongoing change agenda, on ways to distil in a systematic manner the “tacit knowledge” that is generated by operational staff during the day to day processes and practices that occur when designing projects and providing project implementation support in different countries. So as we learned from a presenter during the HD Forum, operational staff should be encouraged to reflect on their experiences and "debriefing mechanisms," either in the form of recorded interviews or specific summary briefs, should be adopted to summarize and disseminate operational "how to” knowledge to management and operational staff alike. This would complement nicely the ex-post impact evaluation work as it would focus on the "how to" processes and practices, including the adaptation and adjustments that are needed in light of changing country realities over the life of a project or program. The recently announced "sabbatical" initiative for operational staff announced by the Development Economics Vice Presidency (DEC) is a step in the right direction as this new program aims to give on-the-ground experts from regions and networks time to reflect on lessons learned from their operational experience, share their knowledge with staff, identify areas needing further research, and collaborate with researchers on how to fill those gaps.

Thanks, Patricio. I completely agree on your point about tapping into and codifying tacit knowledge. I think the DEC sabbatical scheme will help, but it's only a small step in that direction. A.

Submitted by Eva Jarawan on
Thanks Adam for the thoughtful blog - as usual. I just read a story featured on the World Bank China website that illustrates well the points you make. It's entitled "China: Bringing quality health services to rural residents". This is a quote from the article: "In the past, we had to go to hospitals in the closest city when we were sick. They were far and very crowded. It was not easy. Now it is different. Doctors visit us at home and take care of our health. So we get ill less often and hospital visits are less frequent." Guangming Chen a 70-year-old resident in a small village The concept of "doctors on wheels" (bicycles in this case) is one of the innovations of the World Bank-supported Rural Health Project (Shuo Zang is the TTL). Check it out. Make sure to watch the video as well. Here's the link: http://www.worldbank.org/en/news/feature/2013/04/04/china-bringing-quality-health-services-to-rural-residents?cid=EAP_ChinaNewsletterEN_M_EXT

Submitted by Enis Baris on
Thank you Adam. the debate on the science of delivery reminded me of the title of one of your previous blogs referring to the old wine in a new bottle, without of course meaning to denigrate the very much welcome focus on how best we can deliver development assistance to eliminate poverty and improve shared prosperity, but simply to point out to the fact that preoccupation with the "how" question has been with us for quite a while, albeit under different terminology. To illustrate, you may know that in 1980s the clinical epidemiologists had the formula below to distinguish efficacy from effectiveness: Effectiveness = efficacy x diagnostic accuracy X coverage X user compliance x provider compliance in which efficacy is all about the delivery of science and the rest of the formula being about the science of delivery, namely making sure that: (i) we only use interventions with proven efficacy; (ii) the targetted population corresponds fully to the population at risk without any "false positives" or "false negatives"; (iii) all are reached by the intervention; (iv) those who implement abide by the guidelines and specifications of the intervention as designed; and (v) the population covered also adhere to the intervention specifications. The challenge for us all is to compile all the knowledge, tacit or otherwise, and render it accessible to Bank staff, and indeed to the wider development community in a systematic manner so that we all understand the gaps between efficacy and effectiveness, as well as the determinants thereof for continuous fine-tuning to improve the robustness of our interventions despite within- and inter-country variations in political economy and actual implementation.

Submitted by Nikolai Lorget on
Dr Kim said: "We have just not focused on the enormous complexity of delivering health care in a way that keeps people healthy." "Keep people healthy" You say that the World Bank has been doing science of delivery all along. Do you know how much of Bank work in health projects is to KEEP people healthy , as opposed to treating ill people? 10 % vs 90%? Or 50% vs 50%? So what should this proportion be, since the Bank is spending taxpayer money? 90% vs 10% seems to be a superior mix, to provide incentives. Dr Kim is challenging the World Bank to make this part of the science, but you did not include it.

Submitted by Tamar on
Adam, nice blog which triggered a few thoughts. I think that we overdo the distinction between the “what” and the “how” and overestimate how much we know of the what. Just take the human development sectors: we know that to support the poor and vulnerable and to help families manage risks, we need a social protection system but we really do not know what that system should look like in each country. We know that jobs are important for better living standards (and for productivity and social cohesion, as espoused by the WDR) but what policies are needed to generate those jobs will vary by country. We know that teachers are important to deliver a quality education but there is a lot we do not know about what recruitment, pay, training, promotion, accountability arrangements in a given country context will produce skilled teachers willing and able to invest in children in the classroom. We know that countries underinvest in prevention and public health, but we do not really know what it takes to change individuals’ behavior, be they providers or beneficiaries. Are these what or how questions? I do not know. I think in some instances these represent gaps in our “technical” knowledge. But in many instances what is needed is “adapted” knowledge that builds on the societal preferences, the institutional set-up and the underlying politics of the context in question. If this is what is meant by the science of delivery, then I am all for it. But I heard a lot about the management of the process of delivery yesterday --setting targets, monitoring them relentlessly and holding units accountable for meeting them -- and not enough about what it takes to support the generation of the kind of knowledge that delivers development solutions. I think that the Bank is most effective when staff have access to global knowledge, possess the capacity to analyze well the specific country challenges and then come up with tailor made solutions to local problems. There are many instances when we do this well but not systematically. To be at our best every time requires some changes. Here is my list. 1) a robust system of knowledge management that provides timely access to curated knowledge of what has worked in different settings; this system can only be sustained if staff view contributing to the institution’s stock of knowledge as part of their job; 2) getting out of our comfort zone to look beyond technocratic solutions; working with local partners and understanding local institutions and the political context are essential; 3) scope for experimentation, tolerance for getting it wrong and flexibility to adjust as you go; this means a radical change in our appetite for risk and business model so staff spend less time on procurement packages, obtaining multiple clearances and pushing for sectoral solutions and more time with the client, solving (multisectoral) development problems and helping build sustainable institutions. I recall President Kim’s first townhall meeting with staff. He said that the institution’s mission was inspiring, that the committed and skilled staff were its main asset and that what he needed to do was to remove the obstacles so that staff can get on with their business of serving the poor people of the world. Staff are looking to him to deliver on that promise.

Thanks, Tamar, for these very thoughtful comments. A blog post in its own right! I can see that the “what” and “how” distinction is confusing, and perhaps it’s not the most helpful one anyway. I guess the “what” questions range from broad policy through to the content of services delivery; we do a lot on the former (even though as you say we don’t always have the answers) and less on the latter. As you say, a lot of the discussion the last couple of days on “deliverology” was actually structured around a different distinction, namely between policies and programs, on the one hand, and implementation on the other. “Deliverology” as currently conceived is clearly focused firmly on implementation, which begs the question of where the ideas for policies and programs come from (isn’t that part of the science of delivery?). And because governments do the implementation and the Bank supervises, the Bank doesn't have an obvious role in “deliverology” as interpreted up to now. I like your 3-item list but do (2) and (3) point to a bigger role for the Bank in implementation and hence in “deliverology”? A.

Submitted by Cecile Fruman on
Adam, I have been following the very useful discussion that your blog triggered. There are some great ideas that the Change Team on Knowledge and Solutions will benefit from! I particularly like Tamar's 3 points which resonate very much with our current thinking: the need for a robust knowledge and learning system, a collaborative approach to defining and solving problems with clients and a wide range of stakeholders, and fast learning loops that allow for rapid adjustments during implementation. I also like Patricio's very practical suggestions to distilling "tacit knowledege" and engaging in structured learning activities throughout the project cycle. One of the fundamentals of science of delivery is this ability to capture learning throughout the process and react to it quickly. Let me share with you our working definition of Science of Delivery: "A team-based, cross-disciplinary, and evidence-based approach to wicked development problems that delivers results faster". I would love to hear how this resonates with readers of this blog. Could you also send me some information on the DEC sabbatical initiative? Best, Cecile

Cecile,

Many thanks for your comment. Two quick points if I may.

1) I think it's worth distinguishing between better design (of policies, programs and projects) and better implementation. The discussion in the Bank last week on "deliverology" (which seems to have promoted the science of delivery debate) was essentially about improving implementation, and doing so in real time. I have a post on reconciling the two sciences of delivery today on LTD, and extended the ideas in a presentation yesterday to the president, the chief economist, and the chief economist's advisory panel. I don't disagree that the Bank could do more to help better implementation in real time, but I think it's important we don't lose sight of the huge amount of analytic work that's been done – and continues to be done – on design and implementation issues. While this work hasn't been called the "science of delivery", most of it merits that title: it's hard evidence, based on a social scientific theory of change. I get the sense not everyone knows about it, and feels that the "science of delivery" is virgin territory!

2) In thinking about how to get better design and better implementation, it’s worth reflecting on what the analytic work on delivery tells us might be a reason why design and implementation may not be as good as they could be: weak accountability and poor incentives. Are TTLs and managers rewarded sufficiently for projects and policy dialogue that are built upon the best knowledge available? And are they rewarded sufficiently for good implementation? My sense is the answer to both questions is No. If that’s the case, organizational changes that don’t tackle this fundamental issue will fail. DfID recently took bold steps on the first of these, introducing anonymous peer review, with peer reviews expressly told to recommend rejection or “revise and resubmit” for projects that are poorly designed due to a failure to incorporate state-of-the-art thinking on the topic being addressed. Martin Ravallion had a proposal along these lines though the idea could probably be handled within the existing paperwork. He also had a bold idea that would help tackle the second problem – weak incentives to implement well. He and Mead Over proposed that staff acquire “development impact wealth” over their career according to the development impact of their projects; these points would translate into real money, and would continue to accrue even after a new TTL has taken over the project. Of course, other people will have other ideas. My point is that unless the ideas for change address the fundamental issues of accountability and incentives vis-à-vis putting knowledge to use in design and implementation, I suspect our wheels will be just spinning in the sand. 

You asked about the DEC Visiting Expert Program. The details are here. The first round closed on 3/31/13 but I’m sure there will be others. Demand was very strong!

Best,

Adam

Dear Adam and Tamar

Thank you very much for the original blog and Tamar for her comments. I think the WB focus on delivery and its science will be a welcome change for us in the developing world. Also, I think Tamar's 3-pointer on knowledge management, local partners and experimentation are clear directives.

I will also point to Dr. Kim's prefatory comments before the WB talk by Christina Paxson on the role of higher education. I think WB would do a good job if it advised higher education managers in developing countries also to focus on delivery.

I work at an "excellence" institute in India and I have a tough time convincing our management that "relevance" or "delivery" also poses hard research problems. Also, it needs a new pedagogy of engineering. My attempt is at:
http://www.youtube.com/watch?v=G71maumVZ1A

I have also tried to explain this in a paper "Knowledge and Practice for India as a Developing Country". I think the the development of "practice" and respecting it as first-rate "knowledge" is the key to development.

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2210323

Thank and best wishes.

Milind.

Submitted by Hamed Zandian on

Hello
I am a Ph.D candidate of health policy in Tehran University of Medical Science (TUMS), Iran. My thesis field is in health system financing and equity. I want to ask some questions: Can I get your advice during the completion of my thesis?
Thanks for your attention

Add new comment