Measurement: The missing puzzle piece in early childhood education in North Macedonia

Global interest on early childhood education (ECE) is growing as more and more countries realize that investment in the youngest members of society lies at the heart of human capital development. A considerable body of international research highlights the importance of ECE, not just for school readiness but also for later educational and life outcomes. In view of such evidence, the Government of North Macedonia is committed to expanding access to ECE and improving early learning quality.  

The Missing Piece in the ECE Puzzle

In terms of access, the country has made significant strides in improving participation in preschool education. The number of children enrolled in all types of preschool institutions has increased by 11% between 2009 and 2019, which is a considerable accomplishment.

However, despite improvements of access to ECE, the country has no system for monitoring the quality of ECE, thus facing challenges in answering the question: what is the quality of early learning in North Macedonia and how can we continuously improve it?

Providing answers to such a question becomes even more vital considering the potential learning losses caused by COVID-19. Without an appropriate system of quality measurement, it will be difficult to take stock of the impact of the pandemic on children’s learning and well-being and to design suitable responses.  

The Solution

Luckily, this is about to change! With funding from the World Bank through the implementation of the Social Services Improvement Project (SSIP), the Ministry of Labour and Social Policy (MLSP) is in the process of introducing a set of ECE measurement tools called MELQO (Measuring Early Learning Quality and Outcomes) to generate nationally comparable evidence in ECE. Here we describe our journey in establishing this new measurement system in North Macedonia and share some of the lessons learned from MELQO.

The process of implementation of MELQO includes the following four phases: adaptation, consultation, pre-field testing, and national roll-out.

Adaptation. The process began with the formation of a core MELQO team with experts from academic institutions in North Macedonia, representatives from the MLSP and the Bureau for Development of Education, and technical assistance provided by the World Bank. Consistent with the protocols set out by the international MELQO consortium, the core team undertook the following tasks in the process of tool adaptation:

1. A careful and diligent review of the existing pre-primary policy landscape, including curriculum and early learning standards.
2. An inception report which included the framework for the national study and key questions it would answer.
3. A detailed look at what children should be learning and be able to do as well as how teachers should be supporting this development through their practice and the provision of stimulating, safe learning environments.
4. A mapping of the MELQO domains against the national policy and curriculum frameworks so that all items used are fit for purpose and meet the needs of the Macedonian context

Consultation. Once the draft tools were prepared, a MELQO consultative group was formed convening central and local government officials, academia, representatives from civil society, and international organizations working in the domain of ECE. In January 2021, the core team, with leadership from the MLSP, convened a MELQO stakeholder workshop to: i) inform the working group about the status of the instruments; ii) strengthen the tools through broader consultation and input; and iii) build consensus and take-up of the tools and processes. After the workshop, the core team drafted a summary of input and a report that addressed the questions from stakeholders and outlined how the proposals from the stakeholder group will be incorporated. Closing the feedback loop in this way was essential in making sure that the process of consultation was transparent.

Pre-field testing and training of enumerators. With the start of the COVID-19 outbreak we were not able to test the tools within the planned timeframe as field testing was put on hold for several months until conditions allowed for it. Once the MELQO tools were pre-field tested, the team made refinements based on feasibility and relevance.

In parallel to the field testing, a three-day enumerator training for master trainers also took place. Five master trainers of enumerators were trained (in-person and virtually) and are now experts in the use of MELQO instruments. A thorough selection process of master enumerators and high-quality training are critical for ensuring accuracy and reliability of the findings.

National roll-out. In the beginning of 2022, the country started implementing the first ever national MELQO assessment, which will collect data on children’s learning and development outcomes as well as data on learning environments, parent/caregiver knowledge, and teacher attitudes and practices. The data collection process will include a nationally representative sample of children attending pre-school (approximately 2700 children).

Early Lessons Learned

There are valuable lessons we have learned so far, which we think are relevant in any given context where MELQO or similar assessments are to be implemented. These include:

1. Make sure that there are enabling conditions for measurement at system level.
2. It is critical that government and local stakeholders lead the effort from the very beginning.  
3. The measurement tools need to be developed through a consultative process that engages a range of stakeholders. This is a critical element to build stakeholder buy-in in the process and to improve the quality of the measurement exercise.

So, what are you thinking about doing in your country to better understand ECE quality?