Financial incentives in health: supply- vs. demand-side. Your help is needed!
This page in:
A blogpost on financial incentives in health by one of us in September 2015 generated considerable interest. The post raised several issues, one being whether demand-side financial incentives (like maternal vouchers) are more or less effective at increasing the uptake of key maternal and child health (MCH) interventions than supply-side financial incentives (variously called pay-for-performance (P4P) or performance-based financing (PBF)).
The four of us are now hard at work investigating this question — and related ones — in a much more systematic fashion. And we'd very much welcome your help.
A motivating chart from a quick-and-dirty analysis
The September 2015 blogpost highlighted just 3 recent evaluations – two PBF and one demand-side intervention. After the post, one of us did a quick literature trawl to unearth other studies that estimated impacts of demand- and supply-side side financial incentives on MCH outcomes in a credible way. The results were presented at a symposium on Innovations in Global Health Financing held at Erasmus University Rotterdam in November 2015. The evidence base included 5 PBF schemes and 7 demand-side schemes, including 3 voucher schemes and 4 CCT schemes. On average, the PBF schemes have much smaller impacts on the uptake of MCH interventions than the demand-side schemes – see the chart below.
Getting more systematic
The chart is food for thought, but raises lots of questions. Do the results hold for all studies on the topic? Are there systematic differences within each class of program? For example, do CCT schemes, on average, have larger impacts than voucher schemes? Do PBF schemes with some features have larger impacts than those without them? And are they so much more effective that on average they have larger impacts than demand-side interventions? What about cost-effectiveness?
We’re now well into a systematic review and meta-analysis aimed at answering these and related questions. The details are registered in the PROSPERO International Prospective Register of Systematic Reviews.
Here’s a quick summary of the boundaries of the study:
- We’re focusing on MCH outcomes and uptake of MCH interventions, following the official and supplemental MDG indicators
- We’re only looking at low- and middle-income countries
- Under ‘supply-side incentive schemes’ we’re including PBF schemes that aim to improve MCH intervention coverage rates by financially incentivizing individual health workers or entire facilities
- Under ‘demand-side incentive schemes’ we’re including schemes that are aimed at households. We include: CCT schemes with MCH components; MCH voucher schemes; contributory health insurance schemes covering MCH services; and reductions or removal of user fees for MCH services
- Sometimes, of course, a program or intervention may combine supply- and demand-side incentives, combining, for example, PBF with vouchers. In such cases, studies may not necessarily be able to estimate the effect of each component; sometimes, they may, however
- We only include studies that estimate impacts using one or more of 4 methods. We’re including randomized experiments (individual and cluster level RCTs), as well as quasi-experimental studies using Regression Discontinuity, Instrumental Variables, and double- or triple-differences on repeated cross-sections or panel data
- We’re excluding studies of non-randomized interventions using single-difference evaluation designs, such as cross-sectional with-and-without comparisons, before-and-after comparisons without a comparison group, and interrupted time-series analysis
- We’ll be summarizing the studies in a qualitative fashion. But our main interest is in the meta-analysis we’re planning where we’ll be able to see how (and, hopefully, why) average effects differ between demand- and supply-side interventions, and within each class of intervention
To make sure we’re coming up with meaningful results, we need to be sure we are including all the studies that meet our inclusion criteria. We’re systematically trawling through three types of database:
- Previous systematic reviews – there have been a number, including 5 on PBF and no less than 13 on CCTs
- Curated academic databases, like EconLit, EMBASE and SCOPUS
- Google Scholar – we’re anticipating finding some that weren’t discovered in the two aforementioned databases
In addition, we’re searching through websites of donors and international organizations, and think tanks. We also stumble on studies while reading. Last, we get sent studies by people who know we’re doing this study.
And this is where you, the reader, comes in. If you know of studies that meet our criteria, we’d love to hear from you. You can just paste the details in as a comment below. Or if you prefer to do it privately, just send an email to [email protected].
Thank you!
Dear Sven & Adam,
We will share this announcement on the PBF Community of Practice. One small remark: I am not sure that the divide demand/supply side is so clear-cut. For instance, a voucher scheme is both a demand and a supply scheme: the voucher removes the user fee/transport/information barriers faced by the user but it also rewards the provider on an output-basis. A similar situation can prevail with a free health care policy: in some countries, the facilities are compensated for the income loss by inputs (more staff, more drugs), but in a growing number of countries, the facilities are compensated on an output basis, sometimes with a nice margin for the facility. This was for instance the case of the SONU subsidy in Burkina Faso. But I guess this can be handled in your meta-analysis.
Many thanks, Bruno.
Thanks for sharing with the PBF COP -- we look forward to their input.
Agree very much with your point. As we say in the post: "Sometimes, of course, a program or intervention may combine supply- and demand-side incentives, combining, for example, PBF with vouchers. In such cases, studies may not necessarily be able to estimate the effect of each component; sometimes, they may, however." Of course, we'd like also to show the effect of combining the two sets of incentives. We hope to do this in the meta-analysis. Whether it proves possible will depend on the richness of the evidence base.
All the best,
Adam
Thanks Adam.
I understand that you will address this in your systematic review – but the figure illustrating the blog does not do it ;-)
A voucher scheme is a PBF+ and a PBF- scheme. By the +, I mean that it includes the output based payment component to the provider of the PBF scheme (to address the barrier on the provider side), but it ‘augments’ it by a demand side component often addressing the three key barriers faced by the user (information, transport and user fees). By the -, I mean that the voucher, by design, focuses on a narrower list of services than PBF schemes (which try to cover the full package of services provided by the health facility).
By concentrating the funding on addressing all the barriers for a few specific services, vouchers are designed to beat PBF on their target… and on the specific metric you will use for your own review. The reverse would be a real puzzle to all of us.
My point is that you may want to reconsider your categories. You should probably use at least three main categories : financial incentives to providers only (most PBF schemes), incentives to users only (CCT; some user fee removal schemes), incentives both to providers and users (voucher schemes, Health Equity Funds, many free health care policies).
Another way would be to have even more categories (as we address at least barriers at four levels, we can imagine many combinations).
Best
Bruno
Hi Bruno,
Thanks. I think we all agree -- pls see reply to Anna Marriot's comment below.
Cheers,
Adam et al.
Ces articles pourront peut être intéressant dans le cadre de vos préoccupations;
The assessment of job satisfaction for the healthcare providers in university clinics of Lubumbashi. The Pan African medical journal, 19.
et Fox, S., Witter, S., Wylde, E., Mafuta, E., & Lievens, T. (2013). Paying health workers for performance in a fragmented, fragile state: reflections from Katanga Province, Democratic Republic of Congo. Health policy and planning, czs138.
Bonjour Henri,
Merci de nous avoir communiqué ces deux articles qui sont en effet très intéressants. Nous allons les étudier avec attention.
Bien à vous,
Damien
Hi Adam, this systematic review and meta-analysis will be a great resource and many thanks for the update. This year a HNP working paper reported results from a mixed methods study of a maternal health voucher program in Uganda.
https://www.researchgate.net/publication/306262813_Increased_Coverage_o…
A 2014 Pop Council list of resources, including many maternal health voucher studies, is available on the RBF website:
https://www.rbfhealth.org/sites/rbf/files/Event/Vouchers-Recommended%20…
A maternal health voucher review was published in 2013:
http://www.jhpn.net/index.php/jhpn/article/%20view/2362/996
We're in the midst of a 3ie study of user fee removals for maternity services in Kenya, comparing time trends in delivery locations across multiple DHS rounds. As part of that study this past year, another round of household surveys was conducted in voucher and non-voucher communities previously surveyed 2010 and 2012 to now compare vouchers to free maternity services. Look forward to results next year.
Regards,
Ben
Many thanks, Ben.
We'll chase down the resources you mention, and look forward to the results of your Kenya study, which sounds most interesting.
Best wishes,
Adam
Hi Adam, Damien, Jed and Sven
Thanks for this interesting blog and graph. Very much looking forward to reading the results of this study--some sources you may also want to include as you trawl through the literature, if not included already, are:
1. PubMed
2. Cochrane Reviews (they have several on vouchers)
Also a couple of questions
1. Any reason for not included interrupted time series designs?
2. Within demand side, any thoughts at also looking at unconditional cash transfers to see if any spill over effects on MCH? This has been somewhat studied and is especially relevant in fragile states where unconditional cash transfers are being discussed at present.
with thanks
Aaka
Hi Aaka,
Thank you very much for your feedback, highly appreciated!
We try to cast a wide net with our literature searches and use medical, social science as well as interdisciplinary databases, including PubMed and Cochrane Reviews.
We exclude interrupted time-series studies because these interventions often take place in very dynamic environments, which makes us nervous about strong assumptions about underlying trends.
The comparison with unconditional cash transfers you suggest would indeed be very interesting. But to keep the scope of our review manageable we decided to limit ourselves to interventions that affect the monetary price of delivering or using care directly. Our discussion section will certainly also refer to the unconditional transfer literature.
Cheers,
Adam et al.
Hi Adam, thanks for this and we look forward to the outcomes of this very interesting research. I am also a bit puzzled though about the distinction between demand and supply as it does feel like you are trying to conclude whether one is better than the other. However, often their objectives are entirely different. A supply side incentive may improve performance of the staff and services but may not significantly lead to improved uptake if access barriers like fees/co-payments remain in place for patients. It seems there needs to be some control for the access barriers for patients with any method intended to increase uptake if you are to undertake a fair analysis of their performance. Thanks again.
Hi Anna,
Thank you very much for your comments.
Per our reply to Bruno's point, we recognize that one option -- often encountered in practice -- is to provide financial incentives simultaneously on the demand and supply sides.
We are therefore carefully recording the design and implementation details of the various programs, so that, for example, a voucher scheme that incentivizes both the demand and supply sides will get classified in the meta analysis as a demand-and-supply incentive scheme. One interesting question is whether there are synergies between supply- and demand-side incentives, so that the effect of having both types of incentive simulatenously is larger than the sum of the effects of supply- and demand-side incentives implemented alone.
Cheers,
Adam et al.
Hi Adam, thanks for this post and for your work. The project looks very promising and I am sure it is going to be a great resource for many researchers and practitioners. You might want to have a look at this recent paper as well, focusing on the supply-side: http://cepr.org/active/publications/discussion_papers/dp.php?dpno=11515
All the best.
Thank you very much, Andrea -- highly appreciated!
Cheers,
Adam et al.
Hi Adam et al.,
Very interesting research. I have a couple of comments. I have been doing work on access to MCH services and financial protection in low-income and middle-income countries. Another program that could be considered on the demand-side is the provision of personal ID and registration of the beneficiaries, they might have a confounding influence on the relationship between the financial incentives and access to MCH interventions. Registration and ID have been part of health insurance, CCT and some voucher schemes covering MCH services. In a recent research paper, we found that ID cards that linked the entitled services to identifiable members emerged as a very strong predictor of access to integrated maternal delivery care (hospital plus PHC).
Regarding the options, joint demand-and-supply side incentives are missing from your graph. I think they should be given some attention. For example, if P4P incentives were combined with the provision of IDs to beneficiaries, the two might emerge stronger and have a larger effect than if each program was implemented separately.
Best, Veronica
Hi Veronica,
Thank you very much for your comments. Could you kindly send Sven the paper on registration / ID scheme impacts ([email protected])?
On the issue of combined supply- and demand-side interventions, per our comments above abd in the original post, we very much agree, and will try to take possible interactions into account in our meta-regression analysis.
Cheers,
Adam et al.
Dear Adam, Damien and Jed
Congratulations on taking on this effort!
Elena Bardasi and myself conducted a systematic review of the gender impacts of SSN two years ago. We should have captured all the IE of CCT published on or before June 2014 that met our criteria. Here is the link to the report and the appendixes with detailed info on the methodology we used and the criteria for IE inclusion:
https://ieg.worldbankgroup.org/evaluations/social-safety-nets-and-gende…
Elena and I would be happy to share the IE coding database with you if useful. Just let us know and we send it by email.
Best,
Gisela
Hi Gisele,
Many thanks indeed. We will definitely read your SR. And yes pls, we'd love to have the database -- pls could you send it to Sven at [email protected].
All the best,
Adam