Syndicate content

Deliverology and all that

Shanta Devarajan's picture

As a student of service delivery, I was delighted to read about Sir Michael Barber’s effort to conceptualize the implementation of service delivery policies—what he calls “Deliverology”—a problem many of us have grappled with for a long time.  These problems are widespread:  20 percent of 7th grade students in Tanzania couldn’t read Kiswahili (and 50 percent couldn’t read English); the latest ASER report in India shows that learning outcomes are declining while enrolment is rising;. Similarly, doctors in rural Senegal spend a total of 39 minutes a day seeing patients; in India, unqualified private-sector doctors (otherwise known as “quacks”) appear to provide better clinical care than qualified public-sector doctors.

The question is:  Can Deliverology in principle help solve these problems?  Conceptual approach.  Deliverology is based on the notion that traditional public-sector organizations are not geared towards delivering results—such as student learning outcomes or quality clinical care—for several reasons.  The organization’s goals are too many and too diffuse.  Frequently, the goals cannot be quantified.  For those that can be, there is very little real-time data to monitor progress towards the goals.  As a result, staff and management within the organization do not work towards these goals.  Rather, they may try to maximize the size of their unit or the budget under their control.



To overcome these constraints, Deliverology proposes that a small, high-quality delivery unit be established, reporting directly to the head of the organization, that is charged with championing the delivery of a few, well-specified results.  The unit will gather and report real-time data on progress towards the results; work with the line managers in the system to make mid-course corrections; and set up routines where the leader and the stakeholders can review performance and make decisions.  

Both the diagnosis and proposed solution are compelling.  But there are at least two other important elements of service delivery that are particularly salient in education and health.  

  • The client: By “clients” I don’t mean the government.  Our real clients are the households and children who will benefit from better service delivery. Student learning outcomes are the result of quality teaching and effort by the student (and his or her family).  There is now considerable evidence that parents’ participation in their children’s education can have a significant impact on outcomes.  The EDUCO program in El Salvador (where the number of times the parents visited the classroom had a strong association with student test scores); RECURSO in Peru, where parents tested students’ reading ability at home; the famous Uganda study where an information campaign (that empowered communities) led to a sizeable decline in the leakage of public funds for education; and the numerous cases (including in Punjab, Pakistan)  of parents’ paying to send their children to private schools despite the presence of a free, public school all testify to role that families play in promoting learning outcomes. 

    Furthermore, client participation—or, as we called it in the 2004 World Development Report, “client power”—can be a substitute for public-sector monitoring of the delivery of these services.  No matter how well the delivery unit is performing, it is difficult for the government agency to monitor teacher or doctor presence (let alone the quality of service provision) in remote rural areas.  But there is someone else who can:  the student in the classroom or the patient at the clinic.  In a randomized control trial, publishing information about the quality of clinics in Uganda was significantly associated with a reduction in infant mortality.  Giving parents a choice of schools (and information about the schools) has been shown to improve outcomes in a variety of settings—from Bogota, Colombia to Bangladesh to Pakistan.  

    Deliverology assumes that the binding constraint is the behavior of the public servants.  But when it comes to learning outcomes and clinical care, clients’ behavior could be just as important.  And empowering clients—rather than the members of the delivery unit—could do more for changing the behavior of frontline service providers.
     
  • Politics:  The second element that seems to be missing is the fact that the dysfunction in public sector organizations has its roots in politics.  Why are teachers absent about a quarter of the time in India and Uganda?  Why are doctors absent about 40 percent of the time in India, or spending 29-39 minutes a day seeing patients in Senegal and Tanzania?  The naïve answer would be that these public sector officials are paid a salary regardless of whether they show up for work; and monitoring is lax.  The solution would propose changing the way they get paid (perhaps based on presence) and improved monitoring. 

    In most cases, these technocratic solutions won’t work because teachers and doctors are powerful political forces in the community.  Quite often, teachers run the campaigns of the local politicians, in return for which they get a job for which they don’t have to show up.  In Uttar Pradesh, 20 percent of the state legislature are teachers, and another 20 percent are former teachers.   And there is emerging evidence that technocratic solutions fail.  An experiment to introduce time-stamp machines for health workers in Rajasthan never got off the ground: the day before, all the machines were vandalized.  In Kenya, a randomized control trial that introduced contract teachers under two administrative arrangements—one run by the government, the other by an NGO—improved student outcomes only under the latter arrangement.  There is anecdotal evidence that powerful teachers’ unions undermined the implementation for the government-run experiment. 

    In short, the problems that Deliverology seeks to solve—weak incentives for service delivery—are the symptoms of a deeper problem: the capture of the rents in public spending on health and education by service providers who, facilitated by political elites, are able to resist efforts at genuine reform.  Unless this deeper problem is addressed, Deliverology risks bringing about only cosmetic change.

Empirical evidence.  A natural response to what I’ve just said is:  while it may have serious shortcomings, Deliverology seems to be working in Punjab province of Pakistan, where the application of the approach since 2011 has shown substantial improvement in enrolments and the proximate determinants of learning outcomes.  But in the evidence presented, all of the indicators are improvements over time.  None answers the question: what would these indicators have looked like in the absence of the intervention?  While it is possible to think that they would not have improved by as much, there is no corroboration of this proposition.  Furthermore, as Jishnu shows in the next post, even some of the indicators are questionable.

Finally, we now know that it is possible to do rigorous empirical work with well-defined counterfactuals even in fragile situations like Pakistan. In fact, the last decade has seen an explosion of  high-quality education research in Pakistan with clear evidence on the long-term impact of girls’ schooling, the effects of setting up publicly funded private primary schools, the impact of information, the effects of providing more money to public schools through school councils and the short-term effects of test-score accountability. Scaling up those interventions that have been shown to work in this rigorous manner is the way forward. 

Comments

Submitted by vanessa andris on

Shanta, thank you for the insight and courage you convey through this blog. They say the first step in solving a problem is being able to name is accurately. I think you called it here, and I hope that in all the change work we do, we will be inspired to take on underlying issues rather than getting caught up in perpetually treating symptoms. Thanks again for leading the way.

Submitted by Sandeep on

Dear Shanta (and Jishnu), thank you for injecting this welcome realism into the discussion on deliverology. For sure there is the question of tackling bureaucratic inertia, but there's a reason why it is so much more deadly in the delivery of education and health than in crafting monetary or fiscal policy, arising as it does from a lopsided ecosystem of politically powerful unions, financially powerful insurance companies, hapless clients, controlled information flows, and, yes, flawed bureaucratic incentives. And that's just in the US. Add to it the deeply hierarchical societies and near absence of voice and accountability in many of our client countries and the problem becomes all the more complicated.

And on evidence – it seems that variants of deliverology have been tried on parts of the US. What has been the verdict on those?

Submitted by Jishnu Das on

Thank you for your comment. We agree that at first glance there are many seemingly intractable problems. But first, lets give kudos where its due. We know from the cases of Brazil, from Bihar (in Jishnu's slide presentation) and from Punjab, Pakistan between 2003 and 2010 (also discussed in Jishnu's presentation) that change can come, and it can come rapidly. We are also deepening our understanding of how change comes, and one of the lessons is that there is no unique recipe. In some situations an unusually astute politician makes service delivery part of their platform and turns out to be exceptionally adroit at building the right coalitions (Bihar). In another, rising anxiety and distress about educational costs and inequality leads to a national dialogue (Chile). In others, households start using "exit" options (the private sector and charter schools in Pakistan, Northern India and the U.S.) and in yet others, fundamental changes in education structure (Poland) leads to large changes in test-scores (Merilee Grindle's book, "Against the Odds" documents more examples of education reform working in Latin America). In each of these cases, perhaps better management plays a role, but our understanding of the previous instances of deliverology is that the impacts have yet to be carefully studied and there is considerable debate in the U.K. of whether the overall approach of centralized monitoring and target setting leads to better outcomes, or simply better single-minded adherence to targets! It is not surprising that U.K. was where this all started. After all, in 1862, U.K. was also the first country to institute the "revised code" for education, commonly called "Payment for Results". Under this scheme, school inspectors would pay bonuses to schools depending on things they found during inspection, and not surprisingly, schools started catering to the inspectors rather than the parents, with disastrous results. Matthew Arnold, the poet (and an inspector), after a visit to mainland Europe, writes "I find in English schools....a deadness, a slackness and a discouragement....This changes is certainly to be attributed to the `Payment by Results' school legislation of 1862". Will the target approach and centralized monitoring lead to similar problems today?

There is an old saying in management: "what gets measured gets done".

The idea of a delivery unit, linked to the head of an organisation seems to be a good way forward.

However, I guess in the examples you quote - 39 minutes for doctors in Senegal, a key difficultly is collecting the correct data. Do you feel that the data is available but not acted on or is it there in the first place?

Shanta,

Really enjoyed your blog and wanted to add a few reflections building on our case study (http://www.africagovernance.org/africa/news-entry/AGI-launches-first-case-study-from-the-Insight-and-Learning-Series/)on Liberia’s 150 day plan that President Sirleaf used to kick-start her second term in 2012.

There’s all kinds of buzz (http://www.odi.org.uk/opinion/7703-jim-kims-science-delivery-role-politics)these days about ‘deliverology’ and the ‘science of delivery’ and a lot seems to depend on exactly how you define them.

I couldn’t agree more on your point that deliverology shouldn’t leave out the client. And an important thing we learned from our case study is that it doesn’t have to. A coalition of Government of Liberia actors deserves credit for the 150 day plan’s success. But alongside government, a local civil society organization, the Liberia Media Center (LMC), tracked and publicly reported on the progress of the 150 day plan which pressured the government into an accelerated and highly successful final 50 days of implementation. In many cases the LMC actually improved the government’s monitoring. Admittedly this isn’t an example of the direct client participation that you’re describing but I think it’s an illustration of how bottom-up accountability and top-down performance management can powerfully complement each other.

Your other main critique is that deliverology focuses too much on the technical and that technical solutions can’t fix problems that have underlying political causes. Again I couldn’t agree more. ODI head Kevin Watkins made a similar point (http://www.odi.org.uk/opinion/7703-jim-kims-science-delivery-role-politics) recently about what’s missing from the science of delivery discussion. But there’s no reason that political savvy can’t be integrated into deliverology. A big lesson from our case study is that the Government of Liberia recognized that the start of Sirleaf’s new term was a moment of political opportunity and they seized it. Sometimes the best approach (for both governments and partners) may not be to “fix” the fundamental political incentives (which is clearly very difficult) but to find ways to navigate it.

Really glad you’ve kicked off this discussion.

Add new comment