With the Incheon Declaration recommending governments to allocate between 15 – 20% of total public expenditure to education, how can governments improve the efficiency of their education systems?
Monitoring activities can help by providing timely and reliable data, while encouraging day-to-day use of data collected. This is the case of Peru’s Semáforo Escuela (School Traffic lights), an innovative school monitoring system that combines robust data collection with monthly data feedback routines, and targets and incentives to engage local authorities, while improving decision-making and accountability.
Conceived in 2015 and inspired by the Prime Minister’s Delivery Unit in the UK and Punjab’s Education Roadmap, Semáforo Escuela addresses key data gaps at the school level and is tailored to the needs of Peru’s decentralized education system, where the ministry of education works together with autonomous subnational governments.
Limited resources and widely varying management capabilities meant that many subnational governments lacked the capacity to effectively supervise and manage their schools. Key information such as at student enrollment or teacher attendance were either missing, unreliable or only available infrequently. This changed with Semáforo Escuela.
Three elements work together to improve management practices: (1) monthly reports provide managers across all levels of government with relevant, reliable and timely data; (2) a structure of targets and incentives make indicators meaningful to managers; and (3) information feedback is coupled with targeted technical support from the Central Government to help low-performing units meet their targets.
Each year Semáforo Escuela will visit more than 32,000 schools, collecting information from more than 250,000 teachers and over 4 million students. After its first year of implementation, great progress was achieved to incorporate evidence into policy discussions and encourage local governments to lead improvement actions. One of the results was improving principal attendance by 6.4 percentage points from the baseline, which translates to approximately 2,000 more principals at their schools at any given day or improving teacher presence in class by 1.6 percentage points, which translates to approximately 20 additional class hours taught by teachers.
What made Semáforo Escuela possible?
Semáforo Escuela deploys approximately three hundred inspectors who make un-announced monthly visits to a randomized sample of approximately 5,000 schools, nationwide. Inspectors use tablets with internet connection to provide monthly snapshots on indicators that focus on staff and student attendance, school management and the availability of structural inputs (i.e. textbooks, infrastructure, water and sanitation, etc.). Data is mainly collected through observation, documentation review and interviews with the teachers and principal.
The sample is representative at the three different levels of government (national, subnational and school districts), by area (urban/rural), and school level (primary/secondary). To maximize the impact of follow-up inspections, bigger schools are visited at least twice a year, while others are visited once. By the end of a single year, data on more than 90% of the student population in public schools is collected.
Digital data collection makes Semáforo Escuela fast and adaptative. After each school visit, data can be sent in real-time through mobile internet routers. This reduces human errors in data processing and provides more time for data cleaning and addressing mistakes. Survey questions can also be updated on-the-run and monthly snapshots make the system capable of capturing fluctuations of the indicator throughout the year. This provides managers with valuable information that can help to identify problems quickly, understand seasonal trends and iteratively test their own interventions.
By establishing data feedback routines and setting up a structure of targets and incentives, Semáforo Escuela creates a demand for data to inform day-to-day decisions of managers.
The right data reaching the right people, at the right time
Monthly performance reports are provided to the Ministry of Education and to local authorities at the regional and school district levels. Reports are tailored for each recipient and designed to intuitively communicate key insights and invite comparisons with neighboring school districts, top performers or the national average. Using colored charts and heat maps, recipients can intuitively understand performance data.
Targets and incentives make managers focus on evidence
After baseline values are collected, targets are jointly agreed between the Ministry of Education and its subnational governments. Performance is then linked to financial incentives to drive behavior of authorities. Non-financial incentives are also implemented. Incentives foster healthy competition in the system by creating awareness about what others are doing. Semáforo Escuela data is also publicly available on the Ministry’s website to promote transparency and civil society engagement.
Semáforo Escuela was conceived to not only measure service delivery but also to support their improvement. A specialized “enabling team” was set up in charge of guiding authorities through the utilization of monitoring data and to support targeted subnational governments in the implementation of improvement initiatives. The enabling team promotes coordination between the different levels of government and uses a hands-down approach to provide tailored support to local authorities.
Looking to capture the system’s inherent innovation, the enabling team searched for “good practices” which were defined as simple, low-cost actions that improve indicator performance. To identify good practices, high performing school districts in challenging contexts were identified and visited to collect qualitative data on the processes behind the indicator to find specific actions or approaches which could explain good performance.
In a dedicated section of the monthly reports, successful cases are showcased with a brief but concrete description of the key actions taken to show successful stories and incentivize action.
Conclusion: All three elements drive improvements in the system
Semáforo Escuela is currently in its fourth year of implementation and has consolidated as a reliable source of school data for managers across Peru’s Education system. Understanding the benefits of monitoring and actionable data, some school districts have used their own resources to implement a local version of Semáforo Escuela, that complements the information collected by the centralized system: for example, making additional follow-up visits to schools with poor performance, or visiting schools in targeted areas which were left out by the original random sample.
The system’s success can be explained by its focus on data utilization and the tight coupling of three main elements: (1) monthly reporting routines; (2) a structure of targets and incentives; and (3) technical support provided, while promoting accountability. The way in which all parts of the system come together in each monthly cycle can be synthesized in the following diagram:
Key Takeaways
- Three main elements act together to shift the attention of managers on monitoring data and action.
- Real-time monitoring using technology makes the system responsive and adaptable. Enables the system to identify alerts and take corrective measures quickly.
- Monthly data reporting allows the identification of trends over time. This can improve planning and the understanding of how indicator performance is affected.
- Reliable school data improves management practices by informing prioritization of actions and improvement plans.
- Targets and incentives coupled with technical support from the central government encourage authorities to lead improvement initiatives.
- The system’s dynamic monthly cycles promote intergovernmental coordination.
Ignacio Pezo contributed to the blog. He and Juan Baron are members of the Management Capacity and Service Delivery Thematic Group at the World Bank.
Join the Conversation