This is the ninth in our series of job market posts this year.
Teachers’ attendance can be improved if they are monitored by head-teachers using mobile technology, but only if the associated reports trigger bonus payments.
Policy question
Can high-stakes decentralized monitoring improve civil servant performance, or will it lead to collusion between the monitor and civil servant? And what happens to the quality of information when we raise the stakes of reports?
Absent civil servants are a major problem in many developing countries (Bold et al, 2016). For example, more than a quarter of teachers in Uganda don’t show up to school on a given day, and this number is much higher in rural areas. When teachers rarely teach, it is little surprise that across the developing world students are not learning much at schools.
We know that monitoring, combined with bonus payments, has the potential to improve attendance (Duflo, Hanna and Ryan, 2012), but it is unlikely that centralized monitoring by government can effectively achieve this. In most education departments district inspectors are responsible for monitoring, but they are under-staffed and rarely have the budget to visit schools in person. For example, the average school in Kenya was only visited twice by a district inspector in 2011 (Service Delivery Indicators). In contrast, McDonald’s visits each franchise over 100 times a year for quality inspections. Moreover, merely increasing government monitors may not work: In Uganda the number of inspectors has more than quadrupled since 2008 with no evidence of improvement in teacher attendance.
A possible alternative is to use local monitors like head teachers to report on attendance. Reports can be collected at nominal cost using mobile technology. But if the reports are linked to payments there is a risk of collusion: head-teachers could simply report absent teachers as present, in return for a share of the bonus payment. Indeed, a Kenyan program that rewarded teachers based on reports submitted by their principal did not have any impact on teacher behavior (Chen et al. 2001)
Moreover, there is risk that combining local reports with financial incentives also distorts the quality of administrative data and we are left with bad performance, bad data, and slightly richer teachers.
Can government meet the dual objective of obtaining reliable information and improving performance through paying for locally monitored performance?
Experiment
In my job-market paper, with co-authors Clare Leaver, Ibrahim Kasirye, Pieter Serneels, and Andrew Zeitlin, we answer the question by studying teacher absenteeism in Uganda.
To start, we experimentally varied whether local reports are combined with financial incentives or not. Working with World Vision, head teachers were trained how to report teacher attendance on a mobile phone. In one treatment (hereafter referred to as Info), the reports are sent to a central database, aggregated to school-level, and a summary of the results is re-broadcast back to parents and teachers via SMS. The second treatment (hereafter referred to as Info&Bonus), was exactly the same, except that we also paid a monthly bonus to the teacher of UShs 60,000 (about USD23, or 30 per cent of a monthly salary) if a teacher was consistently marked as present that month. The program lasted from July 2012 to August 2013.
Theoretical framework
Next, we developed a simple bargaining model to show how the preferences of both teacher (agent) and head-teacher (monitor) affect teacher attendance and head-teacher monitoring and reporting, and how this depends on the financial stakes attached to the reports.
The model predicts that raising the stakes of local reports can improve average attendance, even though it introduces opportunities for collusion. A bonus scheme provides opportunities for a head-teacher who cares about teacher attendance to negotiate a contract where a teacher attends in order to receive the bonus payment. This is in line with the more general Coasian insight that bargaining can lead to a more socially efficient outcome. However, if the teacher’s cost of attendance is high and/or the head-teacher places a low value on attendance, then they prefer to collude and the head-teacher falsely reports an absent teacher as present.
The relative size of these two impacts depends on how much teachers dislike attending and how much head-teachers value it.
To evaluate the potential trade-off between performance and quality of information, the model also considers the welfare of a bureaucrat who values teacher presence but makes a costly policy mistake if she holds the wrong belief about teacher absence. Beliefs are wrong whenever there is no report or a false report of an absent teacher. Since both false and truthful reporting increase with the introduction of bonus payments, the model is ambiguous as to the net impact on the quality of information.
Results
Consistent with the theory, we find that monitoring had the largest impact on attendance when it was combined with bonus payments: teacher attendance, as measured by our own independent spot-checks, increased by 9 percentage points – a 27 percent reduction in absenteeism (Figure one). In addition, pupil enrollment increased substantially - between 8 and 14 percent - when monitoring was combined with bonus payments.
We also measured learning outcomes prior and after the program, but because of the substantial non-random change in pupil composition across the treatment arms, we cannot reach any definitive conclusion on changes in learning.
Moreover and somewhat surprisingly, the quality of information actually improved when we raised the stakes. There was collusion - head teachers reported absent teachers falsely reported as present - but this was more than offset by the higher reporting frequency and truthful reporting of the higher teacher presence.
Welfare analysis
Finally, we use these results to undertake a detailed cost-benefit analysis of the alternative intervention designs. To place a financial value on the expected total pupil benefit from improved teacher performance, we back out the implied dropout rate in the different treatment arms. We then calculate the financial returns to an additional year of education in Uganda, using earnings and employment data and previous authors' estimates of the rate of return to education in developing countries. We conclude that for even highly conservative assumptions, it pays to pay for locally monitored performance in the Ugandan context.
Looking ahead
Looking beyond education, there are many settings where public sector organizations do (or could) rely on reports by local monitors that are costly to verify. Can high-stakes monitoring work in other contexts?
Our theoretical framework can provide some insight into this question. Failures in public service delivery can be interpreted as a breakdown of Coasian bargaining. Seen in this light, the question of transaction costs becomes paramount. Our results suggest that performance can be improved using high-stakes local monitoring where:
Jacobus Cilliers is an assistant teaching professor at the McCourt School of Public Policy at Georgetown University. His job market paper is available on his website.
Teachers’ attendance can be improved if they are monitored by head-teachers using mobile technology, but only if the associated reports trigger bonus payments.
Policy question
Can high-stakes decentralized monitoring improve civil servant performance, or will it lead to collusion between the monitor and civil servant? And what happens to the quality of information when we raise the stakes of reports?
Absent civil servants are a major problem in many developing countries (Bold et al, 2016). For example, more than a quarter of teachers in Uganda don’t show up to school on a given day, and this number is much higher in rural areas. When teachers rarely teach, it is little surprise that across the developing world students are not learning much at schools.
We know that monitoring, combined with bonus payments, has the potential to improve attendance (Duflo, Hanna and Ryan, 2012), but it is unlikely that centralized monitoring by government can effectively achieve this. In most education departments district inspectors are responsible for monitoring, but they are under-staffed and rarely have the budget to visit schools in person. For example, the average school in Kenya was only visited twice by a district inspector in 2011 (Service Delivery Indicators). In contrast, McDonald’s visits each franchise over 100 times a year for quality inspections. Moreover, merely increasing government monitors may not work: In Uganda the number of inspectors has more than quadrupled since 2008 with no evidence of improvement in teacher attendance.
A possible alternative is to use local monitors like head teachers to report on attendance. Reports can be collected at nominal cost using mobile technology. But if the reports are linked to payments there is a risk of collusion: head-teachers could simply report absent teachers as present, in return for a share of the bonus payment. Indeed, a Kenyan program that rewarded teachers based on reports submitted by their principal did not have any impact on teacher behavior (Chen et al. 2001)
Moreover, there is risk that combining local reports with financial incentives also distorts the quality of administrative data and we are left with bad performance, bad data, and slightly richer teachers.
Can government meet the dual objective of obtaining reliable information and improving performance through paying for locally monitored performance?
Experiment
In my job-market paper, with co-authors Clare Leaver, Ibrahim Kasirye, Pieter Serneels, and Andrew Zeitlin, we answer the question by studying teacher absenteeism in Uganda.
To start, we experimentally varied whether local reports are combined with financial incentives or not. Working with World Vision, head teachers were trained how to report teacher attendance on a mobile phone. In one treatment (hereafter referred to as Info), the reports are sent to a central database, aggregated to school-level, and a summary of the results is re-broadcast back to parents and teachers via SMS. The second treatment (hereafter referred to as Info&Bonus), was exactly the same, except that we also paid a monthly bonus to the teacher of UShs 60,000 (about USD23, or 30 per cent of a monthly salary) if a teacher was consistently marked as present that month. The program lasted from July 2012 to August 2013.
Theoretical framework
Next, we developed a simple bargaining model to show how the preferences of both teacher (agent) and head-teacher (monitor) affect teacher attendance and head-teacher monitoring and reporting, and how this depends on the financial stakes attached to the reports.
The model predicts that raising the stakes of local reports can improve average attendance, even though it introduces opportunities for collusion. A bonus scheme provides opportunities for a head-teacher who cares about teacher attendance to negotiate a contract where a teacher attends in order to receive the bonus payment. This is in line with the more general Coasian insight that bargaining can lead to a more socially efficient outcome. However, if the teacher’s cost of attendance is high and/or the head-teacher places a low value on attendance, then they prefer to collude and the head-teacher falsely reports an absent teacher as present.
The relative size of these two impacts depends on how much teachers dislike attending and how much head-teachers value it.
To evaluate the potential trade-off between performance and quality of information, the model also considers the welfare of a bureaucrat who values teacher presence but makes a costly policy mistake if she holds the wrong belief about teacher absence. Beliefs are wrong whenever there is no report or a false report of an absent teacher. Since both false and truthful reporting increase with the introduction of bonus payments, the model is ambiguous as to the net impact on the quality of information.
Results
Consistent with the theory, we find that monitoring had the largest impact on attendance when it was combined with bonus payments: teacher attendance, as measured by our own independent spot-checks, increased by 9 percentage points – a 27 percent reduction in absenteeism (Figure one). In addition, pupil enrollment increased substantially - between 8 and 14 percent - when monitoring was combined with bonus payments.
We also measured learning outcomes prior and after the program, but because of the substantial non-random change in pupil composition across the treatment arms, we cannot reach any definitive conclusion on changes in learning.
Moreover and somewhat surprisingly, the quality of information actually improved when we raised the stakes. There was collusion - head teachers reported absent teachers falsely reported as present - but this was more than offset by the higher reporting frequency and truthful reporting of the higher teacher presence.
Welfare analysis
Finally, we use these results to undertake a detailed cost-benefit analysis of the alternative intervention designs. To place a financial value on the expected total pupil benefit from improved teacher performance, we back out the implied dropout rate in the different treatment arms. We then calculate the financial returns to an additional year of education in Uganda, using earnings and employment data and previous authors' estimates of the rate of return to education in developing countries. We conclude that for even highly conservative assumptions, it pays to pay for locally monitored performance in the Ugandan context.
Looking ahead
Looking beyond education, there are many settings where public sector organizations do (or could) rely on reports by local monitors that are costly to verify. Can high-stakes monitoring work in other contexts?
Our theoretical framework can provide some insight into this question. Failures in public service delivery can be interpreted as a breakdown of Coasian bargaining. Seen in this light, the question of transaction costs becomes paramount. Our results suggest that performance can be improved using high-stakes local monitoring where:
- Local monitoring costs are low relative to central government;
- The local monitor shares, at least to some small degree, the preferences of the beneficiaries;
- Transaction costs prohibited bargaining between the local monitor and agent in the first place.
Jacobus Cilliers is an assistant teaching professor at the McCourt School of Public Policy at Georgetown University. His job market paper is available on his website.
Join the Conversation