Four questions to answer before selecting an early childhood development measure

|

This page in:

Image
Children at school
Investments in early childhood education need to be accompanied by better evidence about what works to support children, particularly in low- and middle-income countries. Copyright: Fauzan Ijazah/World Bank

Over the past two decades, early childhood development has become an important policy priority on the global human development agenda. Studies suggesting that improvements to children’s early home and formal learning experiences can have long-term effects on their academic and lifelong outcomes have motivated increased attention and funding for programs and policies designed to help children and their families meet their developmental potential. However, these investments need to be accompanied by more and better evidence about ECD and what works to support children, particularly in low- and middle-income countries.

Monitoring policy goals and targets, studying the effects of ECD policies and interventions, and understanding and assessing the developmental status of individual children all require reliable measurement tools that produce actionable data. In the past, these ECD measurement activities were primarily conducted in high-income countries, and as such, the tools used to measure ECD have traditionally come from high-income countries and validated for use with children from those populations. To address this gap in measurement tools and additional materials , over the last 20 years, there has been the proliferation of dozens of instruments designed to measure various aspects of ECD and its determinants, with low- and middle-income countries and contexts specifically in mind. Technical catalogs of ECD measurement tools used in these countries can be found in recent systematic reviews and guidance toolkits.

As the landscape of ECD measurement tools grows, it is important to remember that there is no “one-size-fits-all” measure suitable for every context, target population, and measurement purpose. Thus, practitioners face a daunting array of choices and can be left with the question: “Which tool is right for my measurement effort?” To address this complex question, the World Bank recently published a short, non-technical guidance note, Guiding Questions for Choosing the Right Tools to Measure Early Childhood Outcomes: Why, What, Who, and How. This note is available in Arabic, English, French, and Spanish.

This note guides users through the critical questions that must be answered to select an ECD measurement tool. Only after defining the goal of measurement, the population of interest, the relevant domains to be measured, and the practical considerations of data collection, can users ensure that they select a tool appropriate for their use. The working paper guides users through these decisions by providing a step-by-step process centered on four critical questions:

1) Clarify the purpose of measurement: the “why”

Clearly defining why early childhood outcomes are being measured and the intended use of data is the most critical step in selecting an outcome measure. Tools can be designed for a variety of purposes, such as population- or system-level monitoring, identifying children at risk for developmental delay, constructive feedback provision and adjustments to classroom instruction, or assessing the effectiveness of an intervention or policy. Misalignment between a tool and its purpose of measurement can lead to invalid use of scores and/or inefficient expenditure of financial and time resources. This could happen, for example, by using a population-level outcome measure to decide whether an individual child is ready for preschool. Because population-level measures are inherently imprecise, they do not capture sufficient detail to make individual decisions.

2) Identify the population of interest: the “who”

After defining the purpose of measurement, potential users need to define “who” the population of interest is. Most experts agree that early childhood spans from 0 to 8 years old, but most tools only focus on a limited age range within this period. In defining the population of interest, users must also decide if they wish to assess the overall distribution of development in a population or focus on a sub-population of children at-risk for delay. Finally, depending on their information needs, users should consider if they want to use a global measure that captures universal, non-culturally specific skills, such those included in the World Bank’s Anchor Items for Measurement of Early Childhood Development, or a measure more closely tailored to the cultural, linguistic, or geographical population of interest.

3) Map the relevant ECD domains or outcomes: the “what”

Once the “who” and the “why” have been defined, users should define the relevant domains of development or outcomes to measure. Domains of early development can include items focused on cognitive, language, psychomotor, and social-emotional skills that children develop over time. Most measurement tools can be “broad but shallow” and capture multiple domains of ECD at a superficial level or “detailed but exhaustive” by concentrating information on a specific area of development.

4) Consider logistical realities of data collection: the “how”

Answering the questions above allows a user to define what data is collected, for whom, and for what purpose. However, an often-overlooked dimension includes the practical considerations surrounding how data will be collected. Assessments are typically administered either directly through a trained enumerator engaging a child in a series of standardized activities or indirectly through an interview with an adult that knows the child well, usually this person being the child’s caregiver or teacher. The sampling strategy of the study, the amount of time, and the financial and human resources allocated to collecting information on ECD outcomes often influence the feasibility of these alternative approaches.

Every measurement situation is unique. Having that in mind, the guidance note reviews five common uses of ECD measures and how these “whys” guide responses to the subsequent questions.
 

 

Population monitoring

Impact evaluation

Research

Formative

Screening

Assessment purpose

(Why?)

Usually, to understand the developmental status of a general population, often to track changes over time

Measure the effect of a program or policy on child development

Generate deeper knowledge of child development and its determinants/ correlates

Help an individual teacher or ECD Facilitator understand the ability of class/group to inform and improve practice

Identify children at risk of disability or developmental delay for intervention

Focus of the sample

(Who?)

Usually, a broad representative sample of a given population

Can be either a representative sample of a broad population or a specific sub- population

Can be either a representative sample of a broad population or a specific sub- population

Usually, a small group of children in a childcare or preschool setting

Can be either broad population or a specific sub- population, often at risk of developmental delay or disability

Children’s age

(Who?)

Determined by the focus of the measurement effort

Determined by the focus of the measurement effort

Determined by the focus of the measurement effort

More often used for preschool aged children

Determined by the focus of the measurement effort

Desired information

(What?)

Usually, holistic scores of early childhood development; in some cases, specific policy- relevant domains scores may be needed

Can be holistic and/or domain-focused depending on the nature of the program/policy under evaluation; many evaluations use multiple measures

Can be holistic and/or domain-focused depending on the nature of the program/policy under evaluation; many research projects use multiple measures

Can be holistic or domain- or skills- focused depending on the objectives of the Teacher/ ECD Facilitator

Usually focused on a specific disability or developmental delay; some screenings are more general

Collection setting

(How?)

Often in conjunction with large household surveys

Center-based and home-based data collection are both common; center- based data collection more common for older children

Center-based and home-based data collection are both common; center- based data collection more common for older children

Center-based data collection more common

Center- and clinic- based collection is common; sometimes included in home- based data collection

Direct/Indirect

(How?)

Indirect more common

Both are common

Both are common

Direct more common

Indirect more common

Timing

(How?)

Annually/semi- annually/regular intervals

Usually at least twice in accordance with program/policy implementation

Usually at least twice in accordance with research questions

Usually conducted multiple times per year

Often conducted at key stages of child development

Data collector

(How?)

Usually trained enumerator teams

Usually trained enumerator teams; some tools require higher capacity enumerators and/or intensive training.

Usually trained enumerator teams; some tools require higher capacity enumerators and/or intensive training.

Usually teachers/ECD Facilitators

Often conducted by trained

paraprofessionals or trained enumerators teams


Deciding on a measurement tool often requires balancing an idealized measurement approach with what is practically and logistically possible. The ECD Measurement team at the World Bank hopes that potential users of ECD outcomes measures will find in the guidance note a useful framework that guides them through answering the “why,” “who,” “what,” and “how” questions that help define the search.

Join the Conversation