One of us – Diana – grew up in Paraguay and never encountered a standardized test in all the years that she went to school. Nor does she know any fellow Paraguayans of her generation who have taken one. Her parents relocated often, so she attended many different schools and, consequently, she experienced a wide range in the quality of schools and teachers. While many of the schools repeatedly failed to achieve the ambitious goals of the national curriculum, some were able to better nurture students. The quality of teaching and the way in which schools are run determine what students learn. Unfortunately, the marked quality differences are generally not evident to students, families, or even teachers, school directors or supervisors because it is difficult to compare learning outcomes in a consistent manner.
Student standardized assessments allow learning outcomes between schools to be compared in a consistent way. In 2015, for the first time, the Ministry of Education and Sciences (MEC) in Paraguay implemented a census-based standardized student assessment through the National Student Achievement Assessment System (SNEPE). Census-based standardized student assessments are tests that are applied to all students in the country from a specific grade rather than to just a random representative sample. Before SNEPE (2015), the country only administered sample-based standardized assessments. The census-based standardized assessments were applied for both math and language (Spanish and Guaraní) and targeted all Paraguayan students in grades 3, 6, 9, and 12. These assessments were a massive undertaking and a landmark in the country’s education evaluation efforts. Almost 500,000 Paraguayan students from 10,000 public and private schools in the country participated in SNEPE (2015), and 4,000 independent test administrators were hired by the government for the application. Because it was done manually, digitizing the results of the 1,500,000 answer sheets took almost two years!
Beginning in January 2018, MEC, with the technical support from the World Bank, designed a dissemination strategy to share the results of SNEPE (2015) with schools and enable them to use the information for the preparation of their school improvement plans. Between April and June 2018, the government rolled out the strategy, which consisted of two parts: (i) the design of dissemination materials and (ii) the organization of workshops with local education authorities and pedagogical supervisors to discuss the dissemination materials.
The dissemination materials included the following:
A school result report that presents the SNEPE (2015) results at the school, district, department, and country level, organized in four sections: (1) average score; (2) box-and-whisker plots summarizing the distribution of the scores; (3) results by performance level; and (4) percentage of correct answers.
An animated short video to help local education stakeholders understand the content of the school result report. The video explains the four sections of the school report card through illustrative examples and on-screen texts. The video has been shared with schools and the school communities via WhatsApp and YouTube.
Four training modules aimed at supporting pedagogical supervisors and school directors in the use of the school report for the preparation and monitoring of their school improvement plans. Module 1 describes what standardized tests are and how schools can use the results to design, evaluate, and monitor school improvement plans. Module 2 offers guidelines for the design of school improvement plans. Module 3 discusses strategies for the evaluation of school improvement plans. Finally, Module 4 explores class observation as an instrument for monitoring school improvement plans.
To introduce the materials to local education stakeholders, the team held a series of workshops across the country. These included seven regional workshops targeting 640 pedagogical supervisors, as well as six departmental workshops in the departments of Alto Paraná and Misiones to cover the training material in more detail.
The workshops served as key dissemination platforms for the SNEPE (2015) results and revealed the vastly different perceptions of the local education stakeholders related to student learning outcomes. During the workshops, participants enthusiastically welcomed the rare opportunity to exchange experiences and collectively reflect on the challenges they face in their daily work. Furthermore, even though many of the technical people from central MEC were under the assumption that the SNEPE standardized evaluation was well understood, it became clear that many participants knew very little about SNEPE’s objective, design, and procedures. Some of the technical people from central MEC were also very interested to hear about the daily challenges faced by the local education stakeholders and how more frequent evaluation might help them monitor their school improvement plans.
A pedagogical supervisor expressed her appreciation during one of the workshops, and said that “in the past, standardized assessments were occasionally administered to our students but then we never heard about the results again.” With the second round of the census-based student standardized assessments administered by MEC in late 2018, we are optimistic that all schools across Paraguay will soon benefit from regular student standardized assessments and the timely dissemination of their results to improve learning for all.
Follow the World Bank Education team on Twitter @wbg_education.
Join the Conversation