Syndicate content

Add new comment

Caution when applying impact evaluation lessons across contexts: the case of financial incentives for health workers

Jed Friedman's picture

These past few weeks I’ve been immersed in reviews of health systems research proposals and it’s fascinating to see the common themes that emerge from each round of proposals as well as the literature cited to justify these themes as worthy of funding.

In this review round, one common theme concerned perceived shortcomings with monetary incentives to public sector workers. Several proposals sought to test financial against non-financial incentives as a means to increase health worker effectiveness. A few other proposals simply posited that financial incentives are ineffective and planned to determine the relative effectiveness only from among competing forms of non-financial incentives. The concern that financial incentives to health workers may ultimately undermine their long-term commitment to provide quality service is a longstanding one rooted in the early psychological literature on extrinsic and intrinsic motivation.

There is, of course, the contrasting view that financial incentives serve to motivate effort (indeed this is a central assumption of neoclassical economics) and this view also has a body of past work to support it. However when it comes to the narrower question of financial versus non-financial incentive effectiveness for public sector workers in developing countries, the existing field-based evidence to support (or refute) either view is not particularly extensive.

One central piece of evidence cited by many of the proposals is a recent working paper by Nava Ashraf, Oriana Bandiera, and Kelsey Jack. Markus introduced this paper in an earlier post. This is a neighborhood clustered randomized trial in Lusaka, Zambia, that enlists stylists at hair salons to promote and sell female condoms. Female condoms are not very popular, and so the health NGO wanted to explore different ways to motivate sellers to increase their adoption.

All participating salons were trained on HIV/AIDS and condom promotion and then randomized into four groups: a “small” financial reward group, a “large” financial reward group, a non-financial reward group who received local recognition for sales and the chance of an invitation to a celebratory ceremony, and a control group.

There is much to recommend in this paper, it is a very careful design that anticipated several potential confounders and dealt with them through actions taken while the trial was still in the field. The study definitively answers the NGO question as to which is the most cost-effective way to increase the distribution of female condoms – the non-financial incentives group wins by a wide margin. In fact sales in either of the two financial arms of the study are not any different from the sales in the control group.

Yet after going through this paper, I don’t think it offers a great deal of guidance for the question of the relative effectiveness of financial vs. non-financial incentives among public sector workers. I say this for two reasons:

1. First the obvious reason – the study population is a sub-sample of hair stylists in Lusaka, Zambia. Over 1200 salons were invited to participate in the study but only a little more than 60% chose to participate through to the end of the study period. The authors are forthright that they study a potentially selected sample of all hair stylists in the city. More importantly for our purposes, this is a study population that does not make a living by providing public services. The introduction of incentives among public sector professionals may have a very different impact than observed here. For the study authors, their cited framework of the popular Benabou and Tirole 2006 paper, which models the effect of incentives on pro-social behavior, is very apt. But the applicability of this framework to public sector professionals is not as clear.

2. I see another difficulty with generalization from the Ashraf et al study. The “small” financial reward group indeed faced a very low powered financial incentive, with a 50 Kwacha payment for each condom pack sold. 50 Kwacha is a small amount of money, even in Lusaka. A stylist would have to sell 60 packages to equal the income from one haircut (the average price of a haircut appears to be about 3000 Kwacha). In contrast, the average participant in this group sold 7 packages in a year. It seems like it takes a great deal of effort to move this product – even those stylists in the most successful, non-financial, group sold an average of 15 packages in year. So 50 Kwacha is indeed a very low powered financial incentive.

It strikes me, though, that the “large” financial incentive, while an order of magnitude larger than the small group, is still not particularly large. Each sale netted the stylist 450 Kwacha (almost the wholesale value of a condom pack – 500 Kwacha). Nevertheless, the annual earnings from this incentive – since this group too sold about 7 packages in a year – totals the income from one hairstyling. While large in relative terms (and probably quite large from the perspective of the NGO) this still doesn’t strike me a high-powered financial incentive if it takes a great deal of effort to sell female condoms (presumably much more effort needed than it does to recruit and cultivate clients for haircuts).

Both financial incentives are of course contrasted with the non-financial group that received public recognition from its sales. But what we don’t know is the subjective value the stylists placed on this incentive – public recognition and other social incentives can be a powerful motivator. But, as in the basic framework of the Benabou and Tirole paper, individuals implicitly contrast the expected returns from non-financial incentives with the returns from financial ones. Since the financial returns in this study are relatively small, it’s very possible that we have, de facto, a test of a non-financial incentive of greater power than either financial incentive.

Now of course my comments do not detract from the main goal of the paper, which is to determine the most cost-effective way to promote female condoms. But the results do not necessarily tell us very much about the relative cost-effectiveness of financial incentives in, say, the Zambia health workforce since it is a different population – one that is already paid to provide health care – that would likely be confronted with greater financial incentives if an incentive scheme were ever adopted by the health system (I know this as I have been part of discussions concerning a pilot program there).

Tying financial incentives to the expected activities of health workers may demotivate them, or may motivate in the short-run but demotivate in the long-run when incentives are withdrawn or modified. Or perhaps financial incentives would serve to motivate and complement intrinsic motivation (imagine a rural health workforce already demotivated by low salaries and inadequate facilities and supplies). We know far to little of the practice to assume financial incentives to health workers will ultimately backfire. Clearly the interaction between the specific contract and the specific context will play a determinative role. The Ashraf et al. paper is one link in the evidence chain, but it is a short chain at the moment.