From data to action: A validation study of an early childhood education measurement tool

High quality early childhood education (ECE) can help improve early childhood development (ECD) outcomes and provide children with a strong foundation for later learning.

However, in many contexts, there is scant information about the quality of ECE services to guide policymakers and other stakeholders. Considering this gap in tools and resources to measure quality in ECE settings, the World Bank released the Teach ECE classroom observation tool in 2021 along with a full suite of resources to support its implementation.

Any ECE quality measurement tool is only useful when it can generate reliable information and guide the valid use of results. Ensuring that a measurement tool is rigorously tested for reliability and validity means that its results can be trusted, essential for making informed decisions about education systems.

A recent published study used a large sample of Teach ECE data from five countries—spanning low-, middle-, and high-income—to explore the technical properties of Teach ECE. In this blog, we summarize the study's key messages and highlight how Teach ECE can improve our understanding of quality in ECE settings globally.

What does Teach ECE measure?

The Teach ECE classroom observation tool was developed to measure process quality features in ECE settings. Process quality refers to how well teacher-student interactions, teaching practices, and experiences in the classroom support children’s learning and well-being. In this sense, Teach ECE produces system-level information on quality teaching practices that could help guide policies, programs, and investments in improving ECE systems. You can read more about the Teach ECE assessment in this blog.

Based on an extensive review of ECE quality literature, Teach ECE conceptualizes and measures the quality of teaching practices (in other words, process quality) into three distinct areas:

• Classroom culture: How the ECE teacher creates a supportive learning environment with positive behavioral expectations for children.
• Guided learning: How the ECE teacher supports children’s learning in the classroom through leading clear and effective learning activities that challenge children to think and include feedback and scaffolded guidance.
• Socioemotional skills: How the ECE teacher promotes the development of social and emotional skills, such as autonomy, perseverance, and cooperation.

What are the study findings?

The study used a total of 1,885 Teach ECE classroom observations from five countries to explore the technical properties of the Teach ECE tool. Using advanced statistical techniques, the paper examined whether the data from these observations reproduced the dimensions of ECE quality as intended.

The results across the five countries show that the Teach ECE tool is true to its original design. While closely related, classroom culture, guided learning, and socioemotional skills are indeed three distinct dimensions of process quality. Therefore, Teach ECE’s ability to produce scores that inform about these separate quality dimensions increases the chances that this tool may meet policymakers' information needs and help them make informed decisions to impact policy and practice.

In addition to investigating the structure of Teach ECE, the study also found that Teach ECE scores are highly reliable. In practical terms, highly reliable scores mean less error in the measurement process, leading to more accurate information regarding the quality of teaching practices.

What are the limitations of Teach ECE?

The study found that Teach ECE scores are sufficiently reliable and have many valid uses, including system-level monitoring, impact evaluation, and research studies. However, it also found that individual Teach ECE scores should not be used for high-stakes decisions about individual ECE teachers or classrooms.

In general, Teach ECE scores can generate aggregated information about teaching practices. Still, more analytical work and evidence are needed before using Teach ECE to assess the practice of individual ECE teachers. This finding is similar to reviews of other classroom observation tools, which conclude that multiple observations of the same teacher on different days, along with other sources of evidence on their teaching practices, choices, and needs, are required to make high-stakes decisions about individuals.

Another limitation of Teach ECE scores is that the comparability of scores across contexts is not fully supported. While the study describes some differences in average Teach ECE scores across countries, further analytical work is required to determine whether some countries or contexts have systematically higher or lower scores on quality teaching practices than other countries.

What does this mean for potential Teach ECE users?

Across five diverse countries, the Teach ECE tool functioned well from a psychometric perspective. Potential users can have confidence that, while standardized, Teach ECE and its results are relevant across global contexts. The reliability of Teach ECE scores is high, and scores are valid for use in systems-level monitoring, impact evaluation, and research. With Teach ECE demonstrating robust performance across different countries, education authorities around the world can now rely on an additional measurement tool and resources to monitor and evaluate ECE classroom practices, providing helpful information on which to base decisions around efforts to improve ECE quality.

To receive weekly articles, sign-up here