Across the world, teacher’s salaries are almost universally determined by educational background, training, or years of experience, rather than performance. Yet a growing body of research shows that these measures are poor proxies for a teacher’s actual effectiveness in the classroom. They show surprisingly little correlation with teachers’ ability to raise their students’ learning.
Research shows that even when teachers have identical formal qualifications and students are randomly assigned across classrooms, value-added learning gains across a single school year vary considerably across different classrooms – even in the same grade, in the same school. There is increasing awareness that pay scales de-linked from results may not only undermine current teachers’ incentives to perform, but also make the profession as a whole less attractive to highly talented individuals that school systems need. With widespread student testing offering more abundant and timely measures of school performance, a growing number of school systems are adopting reforms to link teacher’s compensation more directly with their performance.
One such initiative is a school-level teacher bonus program launched in 2008 in Brazil’s Northeast state of Pernambuco — one of the worst performing education systems in the country at the time. Under the program, state-managed schools set annual targets to improve student test scores and pass rates. This combination simultaneously discourages the automatic promotion of children who are not learning and cuts excessively high retention rates, which is a common phenomenon in Brazil.
The size of the bonus for each school depends upon how close a school comes to achieving its annual performance targets. In the first three years, the reward has averaged 1-2 months’ salary for most school personnel. According to Barbara Bruns, “schools failing to achieve at least 50 percent of their targets receive no bonus, giving the program a stronger accountability edge than most of the other performance pay experiments running in different states in Brazil.”
Using difference-in-differences analysis, Barbara and Claudio Ferraz find that the program has had a positive impact: compared to Northeastern Brazilian states and non-program municipal schools in Pernambuco, the schools which implemented this program have seen a large raise in their learning outcomes over the two years. According to Bruns and Ferraz’s findings, schools with more ambitious stretch targets achieved more progress than similar schools with less challenging targets.
An innovative part of the research is the use of a classroom observation tool to try to analyze exactly what changes in response to performance incentives. As Barbara points out, the theory of action of a performance incentive is that it will stimulate change in agent behavior so as to produce more results that are rewarded. However, there is a striking lack of global evidence on exactly what teachers do in response to incentives linked to student test scores – and whether it is positive (like reducing their absence rates or assigning and grading more homework) or negative (like teaching to the test or cheating).
Given this, and the attendant controversy that surrounds most bonus pay experiences, a key interest of Bruns and Ferraz was to try and gather classroom-level data in representative sample schools to measure observed changes in teacher practice. To do this, they trained more than 60 school supervisors and pedagogical coordinators in Pernambuco in the Stallings classroom snapshot method – a high reliability, low inference monitoring tool and carried out observations in 1,800 classrooms in a representative sample of 300 schools. Moreover, the observation technique has been mainstreamed into the usual practice of the Pernambuco education ministry.
“Looking across schools” says Bruns, “we found schools whose teachers spent the most time on instruction were much more likely to achieve the bonus. And over the two years we have tracked schools, we have seen a system-wide increase in the share of time teachers keep all students engaged. It’s modest, but encouraging.”
Pernambuco’s bonus pay experiment is part of a larger reform effort being supported by the World Bank. This is also the most expensive and innovative element of that reform package and so it is being watched by other states in the country. Bruns and Ferraz have also begun working with the municipal school system of Rio de Janeiro to conduct a parallel evaluation of a new bonus program adopted there. As Bruns and co-author Lucrecia Santibanez concluded in a global review of teacher bonus initiatives included in the book Making Schools Work: New Evidence on Accountability Reforms, specific design features, such as the size, predictability and focus of the bonus – what exactly is rewarded and how it is measured – matter a lot for success.
Finally, Barbara cautions that it is important to monitor these programs over time, as both positive and negative features of performance pay experiences can develop over time and can exert potentially strong effects on their impact. But, so far the Pernambuco results have been encouraging and suggest that bonus pay can serve as a stimulus to more efficient classroom practice and improvement in student learning.
For more information, see Barbara’s book and description of the Pernambuco experiment.
Join the Conversation