Service Delivery Indicators surveys: Are systems delivering for citizens?

The speed with which the spread of the COVID-19 pandemic has overwhelmed health and education systems around the world has forced countries to rethink and rapidly re-adjust the modes of delivery of these critical services.

After almost two years of tackling the pandemic, there is plenty of evidence of locally led innovation and creative approaches. But development and implementation of entirely new schemes spontaneously and quickly comes at a huge cost and governments now face competing demands and very tight fiscal space. More widely implemented and better-quality measurement of the key factors that determine the quality of service delivery in health care and education is now more critical than ever in helping countries emerge from the COVID-19 crisis – having protected the hard-won gains in human capital.

To this end, policy makers need sound evidence on the quality and accessibility of education and health services. Service Delivery Indicators (SDI) surveys help provide this evidence by collecting data at the facility level—in primary schools, clinics, and hospitals—across the public and private sector and in urban and rural areas. SDI surveys provide insights to answer four key questions:

• What do providers know? SDI surveys directly test teachers’ knowledge of the material they are supposed to teach and how they teach it, as well as health care providers’ ability to diagnose and treat common medical conditions.  They are the only surveys of this scale that measure health care provider knowledge comparably across countries.
• What do providers do? Visiting unannounced, SDI enumerators assess the levels of absence and the overall workload among teachers and health workers. These efforts act as an independent audit of the health or education workforce.
• What tools do providers have? SDI enumerators directly observe the availability of basic infrastructure, medical equipment, and essential medicines in health facilities.  In schools, enumerators directly observe and confirm if children have textbooks, classrooms have blackboards, and each school has toilets separated by gender. 
• What are children learning? SDI education surveys measure key education outcomes by testing fourth-graders’ language, math, and non-verbal skills.

Implemented for over a decade, SDI surveys have generated nationally representative data that enable governments and citizens to identify gaps and benchmark progress within and across countries over time.  The SDI book: “The Quality of Health and Education Systems Across Africa: Evidence from a Decade of Service Delivery Indicators Surveys,” joint work of this blog's authors with Ruben Conner and Ciro Avitabile, released on December 16, 2021, showcases how SDI data have been used to shape public debate and policy action in countries across Africa, providing insights on the role of good measurement for reform. As technology transforms data collection in health and education, the report charts directions for a measurement agenda to inform how countries develop policies, design interventions, and deliver essential services.

Levers for health system change: strengthen skills and tools on the front lines

A few important findings about health systems that are outlined in the report include:

• The clinical abilities of health care providers, measured as their diagnostic and treatment accuracy on patient case simulations (clinical “vignettes”), vary substantially by country, but providers at lower-level facilities score noticeably worse. This is particularly troubling given that these facilities are where people make first contact with the health system.
• Providers have a high rate of absence and their workload varies dramatically between and within countries, suggesting that there is room for more frequent supervision and potential reallocation of staff to overburdened facilities.
• Despite decades of investment, many facilities lack the basic necessities, including essential medicines, basic diagnostic equipment, and adequate water and sanitation infrastructure; further investigation is needed into why these gaps persist.

Improving schools: Countries can learn from their own top performers

The report outlines the following key findings about education systems:

• Across SDI countries, fewer than half of fourth graders can recite a simple sentence or perform basic mathematical operations. However, differences in learning are large and some schools and students do very well. Identifying some of these top schools in each country and analyzing how they succeed could help set examples for lower-performing schools.
• Almost a quarter of teachers are be absent during unannounced visits and an even larger share are not present in the classroom during their assigned teaching times. Teacher skills are also low, both in content knowledge and pedagogy. Hiring more teachers and basing that recruitment on skills and talent would help reduce the high ratio of students per teacher and improve learning outcomes.
• Basic school inputs, such as whiteboards and functioning toilets, are not always available for teachers and students despite being critical to the overall schooling experience. For education systems aiming to build back better, improving these basic inputs could be a good place to start. However, improving such inputs is not enough. These efforts must be coupled with other policies that align incentives, increase accountability, and are framed within comprehensive programs.

Driving change with SDIs

SDI surveys have provided the evidence to spur policy debates and accelerate reforms in countries like Mozambique, Togo, and Tanzania. In education, the SDI initiative has supported the development of novel modules to capture socio-emotional skills.  In health, researchers are adapting SDI questionnaires to capture different aspects of quality of care, like clinicians’ knowledge about nutrition and non-communicable diseases or patients' experience and satisfaction with care. Finally, new modules to measure the quality of management and to assess health care providers’ well-being and work environment can offer further lessons.

The insights from SDI surveys offer valuable inputs that can help policymakers to position their countries on the right trajectory towards securing the foundations of sustained, inclusive growth going forward.