Beyond the status quo: Using impact evaluation for innovation in health policy

How do we deliver higher-quality health services in low-capacity settings? 
 
This is the question that we have sought to answer through a long-standing impact evaluation (IE) research collaboration with the Nigerian Ministry of Health and the Bill & Melinda Gates Foundation. The results of this collaboration will be presented at the World Bank on February 8 at Beyond the Status Quo: Using Impact Evaluation for Innovation in Health Policy. This one-day event will bring together policymakers, practitioners, and academics to discuss policy implications and ways to further promote and strengthen capacity for evidence-informed policy.

Our work in Nigeria began in 2007 with an evaluation on the extension of malaria-related health services through non-traditional private sector and community actors. Building on this, we soon expanded to addressing various constraints in the health sector on both the demand and supply side. These constraints are well-known. On the supply side, they include: (i) limited service availability due to the lack of human resources, infrastructure in need of rehabilitation, and unreliable provision of essential supplies such as drugs and midwifery kits; (ii) inconsistent services due to limited health worker motivation and supply chain weaknesses; and (iii) low service quality due to, for example, limited information, coaching, and supportive supervision. On the demand side, users and potential users of health services face access and take-up constraints due, for example, to (i) user fees and transportations costs; (ii) perceived irregularity of services, long waiting times, and poor health worker attitude; and (iii) lack of understanding and valuation of the importance especially of preventive services, such as antenatal care.
 
Beyond the Status Quo: Using Impact Evaluation for Innovation in Health Policy will feature results from large-scale impact evaluations directly tackling several of these constraints. We have learned, for example, that:

• Deploying midwives to underserved areas can improve take-up of facility-based birth, a key intervention in reducing maternal mortality. However, increasing human resources and further supply-side investments do not improve the rate of antenatal care, highlighting the need for complementary action to reduce demand-side barriers to access and take-up. Link to brief.

• The effectiveness of delivering malaria-related primary health-services through partnerships with private-sector patent medicine vendors and community volunteers is related to the underlying quality of existing public primary healthcare facilities. While we do not find that these interventions improve malaria-related health outcomes on average, positive impacts are found in areas with high quality primary facilities.

• Providing information, coaching, and mentoring to primary healthcare facilities can improve the quality of care. Information alone, however, is not effective. Link to brief.

• Giving midwives monetary incentives based on their attendance is effective in reducing midwife attrition. Non-monetary incentives, however, did not have any effect, either by themselves or when provided alongside monetary incentives. Link to brief.

• Delivering behaviour change communication through high quality entertainment (“edutainment”) works to improve knowledge and behaviour relating to sexual and reproductive health, including reducing risky sex among youth. The use of mass media for behaviour change is therefore a promising low-cost, high-return strategy. Link to brief.

• Testing for malaria improves the productivity of health workers independent of test results. In other words, there is economic value in simply knowing one’s health status. Link to working paper. 

 
This work has been conducted in close partnership with Nigeria’s Federal Ministry of Health, whose staff have been involved in setting the research agenda, defining specific questions and interventions, and in implementing the research. We are grateful to them and to our many other partners who share the conviction that we need to move beyond the status quo of poor health outcomes, and that we can do so through innovation, experimentation, and learning.