Recently I was spending some time with a survey firm in Tanzania, pre-testing a survey. I got to talking with one of the folks working at the firm about how they compensated their enumerators. He made it clear that they follow a fixed efficiency wage (i.e. x amount per week, above the market clearing wage) – with some pretty close supervision. On the other hand, I’ve worked with firms that pay a clear piece rate, again with some supervision. While I lean towards the efficiency wage approach, it’s not clear to me which is best.
This boils down to a principal-agent problem, one where effort is unobserved and the outcome may not be apparent until it is too late for you to do anything about it. Let’s take a bit more at the pros and cons of each approach. The fixed efficiency wage approach makes it clear that you value this work and makes the interviewer more cautious to mess things up – not only in individual survey implementation, but in doing things like fudging the overall interview. However, as Dale Whittington points out in a nice discussion of some of these issues, you don’t want to set the wage so high as for the enumerators to think you are an idiot (particularly if you are doing a labor survey). Chris Udry and I adopted the efficiency (fixed) wage approach when we did a survey in Ghana and I think Whittington has a point in that the non-monetary incentives also matter a lot in getting higher quality even when you use this approach. We read all of the questionnaires, pretty soon after the enumerators brought them in, and sat down with them individually when we had questions. We ran repeated rounds, and I think this process helped get the enumerators into a frame of mind where they would help suggest questions or provide feedback on others that weren’t working with more willingness than if we paid them a piece rate. Overall, the right level of efficiency wage, with some close attention and appreciation of the quality of the work the enumerators are doing is perhaps more likely to build a sense of a team – with some potential benefits.
However, the fixed efficiency wage definitely can remove the incentive for speed. So of course, the natural alternative is the piece rate, which I have seen combined with some kind of payment for low error rates (with errors defined as missed questions, messing up skip codes, and other mechanical problems). While this clearly gets you a core set of data in a more timely fashion, it runs the risk of skipping respondents who are more difficult to track down and it cuts down on the enumerator’s incentive to probe (indeed, in the more extreme case when they discover the skip code that gets them out of 10 pages, you can bet that will be a common answer). One variant on the individual piece rate incentive would be to move to team based incentives – but these don’t solve your difficult to find respondent nor probing problems. Of course you could specify these in the incentives – but then you head towards what is a fundamental issue with this approach – you need to spend a fair amount of time specifying a range of parameters for the contract. As an important aside, this all gets rather tricky when you think about competitive bidding for a survey. Since labor costs will be one of the two big costs (transport being the other), a firm could pay a piece rate that is right at the market clearing wage and win on price – but quality will suffer. The main problem is that “quality” at this point is rather hard to quantify – and this makes it hard to convince those overseeing the competitive bidding.
Both the piece rate and efficiency wage approaches have some common elements to consider. The first of these is “trust but verify.” That is, to what extent do you want to revisit households to check the answers on the survey? There are good arguments to do this under both approaches, and they are stronger under some type of piece rate contract. In the case of the efficiency wage, if it’s done in an explicit “I am monitoring you” fashion, then to some degree it undermines the potential team spirit thing you might have going on. But it might be worth it for the strong incentive not to mess up. However, my experience with call-backs is that aside from the clear check as to whether the enumerator ever showed up, a lot of other answers are sufficiently exposed to the vagaries of respondent moods and errors that nailing down enumerator effort is tough (not impossible, but tough). Regardless of the degree of monitoring, one penalty I have seen on almost every survey I’ve worked on is that people get fired for making up data.
The second thing that runs common to most contracts is retention and longer term incentives. In my experience, whether piece rate or efficiency wage, part of the salary is kept until the end (be it explicitly linked to performance or not). This has two obvious benefits: 1) It’s a real pain to train enumerators in the middle of fielding surveys (or to see if that substitute is still unemployed – which may be a bad sign by itself), and 2) it sets up another reward for not committing significant survey malfeasance. Side story: One of my favorite enumerators on our Ghana survey figured out we were holding on to our grant funds in dollars. He also figured out that at that point Ghanaian treasury bills were running at 30 percent while inflation was down to about 11 percent. He asked Chris and I for his survey end bonus early on and put it in T-bills (we had to respect his economic logic over our potential incentives). Now he’s a forensic accountant for the US Army.
Ultimately, the type of contract you opt for and the incentives you put in place will depend on what kind of data you’re after. The more the answers depend on probing, on establishing a good rapport with respondents and careful attention to nuance and detail, the more carefully you want to think about the trade-offs between a piece rate type of approach and a fixed efficiency wage approach. Anyhow, this might be a good place to start a discussion on tips/thoughts/observations – any ideas?
Join the Conversation