Deliverology and all that

As a student of service delivery, I was delighted to read about Sir Michael Barber’s effort to conceptualize the implementation of service delivery policies—what he calls “Deliverology”—a problem many of us have grappled with for a long time.  These problems are widespread:  20 percent of 7th grade students in Tanzania couldn’t read Kiswahili (and 50 percent couldn’t read English); the latest ASER report in India shows that learning outcomes are declining while enrolment is rising;. Similarly, doctors in rural Senegal spend a total of 39 minutes a day seeing patients; in India, unqualified private-sector doctors (otherwise known as “quacks”) appear to provide better clinical care than qualified public-sector doctors.

The question is:  Can Deliverology in principle help solve these problems?  Conceptual approach.  Deliverology is based on the notion that traditional public-sector organizations are not geared towards delivering results—such as student learning outcomes or quality clinical care—for several reasons.  The organization’s goals are too many and too diffuse.  Frequently, the goals cannot be quantified.  For those that can be, there is very little real-time data to monitor progress towards the goals.  As a result, staff and management within the organization do not work towards these goals.  Rather, they may try to maximize the size of their unit or the budget under their control.

To overcome these constraints, Deliverology proposes that a small, high-quality delivery unit be established, reporting directly to the head of the organization, that is charged with championing the delivery of a few, well-specified results.  The unit will gather and report real-time data on progress towards the results; work with the line managers in the system to make mid-course corrections; and set up routines where the leader and the stakeholders can review performance and make decisions.  

Both the diagnosis and proposed solution are compelling.  But there are at least two other important elements of service delivery that are particularly salient in education and health.  

• The client: By “clients” I don’t mean the government.  Our real clients are the households and children who will benefit from better service delivery. Student learning outcomes are the result of quality teaching and effort by the student (and his or her family).  There is now considerable evidence that parents’ participation in their children’s education can have a significant impact on outcomes.  The EDUCO program in El Salvador (where the number of times the parents visited the classroom had a strong association with student test scores); RECURSO in Peru, where parents tested students’ reading ability at home; the famous Uganda study where an information campaign (that empowered communities) led to a sizeable decline in the leakage of public funds for education; and the numerous cases (including in Punjab, Pakistan)  of parents’ paying to send their children to private schools despite the presence of a free, public school all testify to role that families play in promoting learning outcomes. 
  
  Furthermore, client participation—or, as we called it in the 2004 World Development Report, “client power”—can be a substitute for public-sector monitoring of the delivery of these services.  No matter how well the delivery unit is performing, it is difficult for the government agency to monitor teacher or doctor presence (let alone the quality of service provision) in remote rural areas.  But there is someone else who can:  the student in the classroom or the patient at the clinic.  In a randomized control trial, publishing information about the quality of clinics in Uganda was significantly associated with a reduction in infant mortality.  Giving parents a choice of schools (and information about the schools) has been shown to improve outcomes in a variety of settings—from Bogota, Colombia to Bangladesh to Pakistan.  
  
  Deliverology assumes that the binding constraint is the behavior of the public servants.  But when it comes to learning outcomes and clinical care, clients’ behavior could be just as important.  And empowering clients—rather than the members of the delivery unit—could do more for changing the behavior of frontline service providers.
   
• Politics:  The second element that seems to be missing is the fact that the dysfunction in public sector organizations has its roots in politics.  Why are teachers absent about a quarter of the time in India and Uganda?  Why are doctors absent about 40 percent of the time in India, or spending 29-39 minutes a day seeing patients in Senegal and Tanzania?  The naïve answer would be that these public sector officials are paid a salary regardless of whether they show up for work; and monitoring is lax.  The solution would propose changing the way they get paid (perhaps based on presence) and improved monitoring. 
  
  In most cases, these technocratic solutions won’t work because teachers and doctors are powerful political forces in the community.  Quite often, teachers run the campaigns of the local politicians, in return for which they get a job for which they don’t have to show up.  In Uttar Pradesh, 20 percent of the state legislature are teachers, and another 20 percent are former teachers.   And there is emerging evidence that technocratic solutions fail.  An experiment to introduce time-stamp machines for health workers in Rajasthan never got off the ground: the day before, all the machines were vandalized.  In Kenya, a randomized control trial that introduced contract teachers under two administrative arrangements—one run by the government, the other by an NGO—improved student outcomes only under the latter arrangement.  There is anecdotal evidence that powerful teachers’ unions undermined the implementation for the government-run experiment. 
  
  In short, the problems that Deliverology seeks to solve—weak incentives for service delivery—are the symptoms of a deeper problem: the capture of the rents in public spending on health and education by service providers who, facilitated by political elites, are able to resist efforts at genuine reform.  Unless this deeper problem is addressed, Deliverology risks bringing about only cosmetic change.

Empirical evidence.  A natural response to what I’ve just said is:  while it may have serious shortcomings, Deliverology seems to be working in Punjab province of Pakistan, where the application of the approach since 2011 has shown substantial improvement in enrolments and the proximate determinants of learning outcomes.  But in the evidence presented, all of the indicators are improvements over time.  None answers the question: what would these indicators have looked like in the absence of the intervention?  While it is possible to think that they would not have improved by as much, there is no corroboration of this proposition.  Furthermore, as Jishnu shows in the next post, even some of the indicators are questionable.

Finally, we now know that it is possible to do rigorous empirical work with well-defined counterfactuals even in fragile situations like Pakistan. In fact, the last decade has seen an explosion of  high-quality education research in Pakistan with clear evidence on the long-term impact of girls’ schooling, the effects of setting up publicly funded private primary schools, the impact of information, the effects of providing more money to public schools through school councils and the short-term effects of test-score accountability. Scaling up those interventions that have been shown to work in this rigorous manner is the way forward.