This is the 10th in this year’s series of posts by PhD students on the job market.
Despite rising school enrollment and educational spending in developing countries, learning outcomes remain low (World Bank, 2024). One potential reason for this stagnation is the lack of teacher accountability in developing countries (Mbiti, 2016), leading, for example, to high teacher absenteeism rates. To address this, incentive-based policies that focus on improving outcomes – rather than just inputs – could offer a solution.
Such policies can induce teacher effort by attaching financial or non-financial rewards to student outcomes. For example, bonuses for teachers can be attached to their students' performance on test scores, known as teacher performance pay. However, more nuanced incentive schemes can be explored. Targeting the performance of schools instead of individual teachers can help achieve greater learning gains through coaching, peer effects, and motivation. Further, non-pecuniary incentives, such as social recognition, can enhance motivation due to the intrinsic characteristics of the teacher's role.
My job market paper investigates a policy in Brazil that introduced a rank-order tournament among public schools. Known as the "Oscars of Education," this program annually awards top-performing schools and provides financial aid to low-performing ones.
The Policy
The Prêmio Escola Nota Dez (which translates to School Score A Award) is a yearly tournament among public schools in Ceará, Brazil. Every year, up to 150 best-performing schools are awarded in the second, fifth-, and ninth grades. These schools receive an extra budget per student and a recognition prize in a ceremony at the state capital. Highly anticipated and celebrated, the ceremony features important political figures of the state.
Figure 1: Awardees celebrating their achievements at the "Oscars of Education" ceremony.
Source: Department of Education of the State of Ceará
Further, up to 150 low-performing schools in the state are designated to the support program, in which they are allocated a budget equivalent to half of that granted to the awarded schools. At first, awarded and supported schools are entitled to a share of the total potential funding. The goal is to pair each awarded school with a supported school. This pairing allows them to develop a cooperation program to exchange pedagogical practices over the subsequent two years. Ultimately, if both schools achieve their performance targets, they receive the remaining share of the funding.
My study aims to comprehend the role of incentives in two aspects First, I estimate the effects of being exposed to the tournament on the school's performance, called the Tournament Effects. Here, I aim to quantitatively assess whether the tournament's introduction triggered learning gains among the eligible schools.
Further, the award could trigger ex-post effects on the schools awarded or supported annually. Receiving social recognition and extra inputs can motivate teachers to enhance future students' learning even further, which I denominate the Prize Effects. In the same fashion, receiving financial support and participating in the cooperation program could affect future students' performance, which I call the Support Effects.
Data
The primary data source in my study is a panel dataset at the school level with test scores from a standardized evaluation conducted by the state. It is a low-stakes evaluation, as it does not influence a student’s GPA or graduation. For this study, I use annual data from 2008, the first available year, until 2019. To estimate the tournament effects, I rely on data for the ninth grade, which mainly consists of around 4000 schools throughout the state. Further, due to limitations in the dataset, I used data from the fifth grade from the same group of schools to examine the prize and support effects.
The Tournament Effects
In order to assess the effects of introducing the policy among eligible schools, I explore two central components in the policy setting. First, schools need at least 20 students enrolled in the evaluated grade to be eligible for the tournament. Second, the tournament for the ninth grade was implemented in 2015. The idea is to compare eligible and non-eligible schools around the 20 students' cutoff before and after the policy was introduced.
This method is denominated Differences-in-Discontinuities (Diff-in-Disc). Intuitively, I use the yearly regression discontinuity estimate as the outcome of a Differences-in-Differences (DiD) analysis. With that, I can relax the continuity assumption of a typical RDD, but now it is required that observations around the cutoff are on a local parallel trend in the absence of treatment.
Diff-in-Disc estimates show that eligible schools had local increases in their scores of around 0.23 and 0.34 standard deviations (SD) for Portuguese and Mathematics, respectively. More intuitively, applying the multiply by 37 rule of thumb (von Hippel, 2024), for a student who would score at the median without the policy, being exposed to the tournament will raise their percentile rank by 8.5 and 12.6, respectively.
These results can be visualized in Figure 2, which plots the yearly regression discontinuity design estimates around the 20 enrolled students' cutoffs. Before introducing the award, there were no statistically significant discontinuities in the scores at the cutoff. After 2015, there has been a constant increase in the size of the discontinuities for both Portuguese and Mathematics.
Figure 2: Yearly regression discontinuity estimates
These results show that schools benefited from competing in the tournament and, in the medium term, increased their test scores performance due to the policy. I conducted a cost-effectiveness analysis (CEA) in line with the general framework proposed by J-PAL (Dhaliwal et al., 2013). For every 100 USD spent on the policy, there was an increment of 2.84 SDs in students' test scores.
The Prize and Support Effects
Further, I estimate the impacts of being awarded and supported by the policy. The aim is to assess whether future cohorts benefit from the award or the support received by the schools. To estimate the prize effects, I explore the rank of the worst-performing awarded school as a cutoff for a fuzzy regression discontinuity design. By comparing schools on both sides of the threshold, I can assess the outcomes of schools that got awarded with those that did not receive the award by a small margin. For the estimation of the support effects, I conduct a similar procedure. However, the cutoff in this case is the rank of the best-performing supported school.
Being awarded has no impact on test scores three years later. However, receiving the support leads to a 0.16 and 0.22 SD increase in Portuguese and Mathematics test scores. The results highlight the positive impact of giving aid to low-performing schools, both financially and pedagogically. Supported schools benefit from the cooperation program, while the awarded schools receiving the financial and non-pecuniary prizes do not alter their future scores.
Policy Implications
Introducing policies that focus on learning outcomes instead of focusing solely on input-based policies can mitigate problems related to the lack of teacher accountability. My job market paper shows the benefits of implementing incentive schemes at the school level, leveraging both financial and non-pecuniary incentives. Further, low-performing schools can be motivated when directly incentivized, as was the case in the studied policy through the support program.
These findings highlight the importance of going beyond bonus programs for teachers to increase learning outcomes in developing countries. First, setting group incentives for schools can help achieve further productivity gains through peer effects. Further, non-pecuniary incentives can foster cooperation and motivation within public schools and ultimately trigger intrinsic social values related to the work as an educator.
Casemiro Campos is a Ph.D. Candidate at Goethe University Frankfurt.
Join the Conversation