Collecting data in education can be a tricky business. After spending considerable resources to design a representative study, enlist and train data collectors, and organize the logistics of data collection, we want to ensure that we capture as true a picture of the situation on the ground as possible. This can be particularly challenging when we attempt to measure complex concepts, such as child development, learning outcomes, or the quality of an educational environment.
Data can be biased by many factors. For example, the very act of observation by itself can influence behavior. How can we expect a teacher to behave “normally” when outsiders sit in her or his classroom taking detailed notes about everything they do? Social desirability bias, where subjects seek to represent themselves in the most positive light, is another common challenge. Asking a teacher, “Do you hit children in your classroom?” may elicit an intense denial, even if the teacher still has a cane in one hand and the ear of a misbehaving child in another.
In recent weeks, several articles have appeared in the main U.S. newspapers– including the Washington Post and the New York Times – discussing the potential benefits and pitfalls of the Los Angeles Times’ decision to publish performance data on individual teachers. Together with an economist, LA Times’ reporters used long-existing data on student test scores by teacher over time, to estimate individual teachers’ “value-added”, that is, the change in a student’s test score in the year that they had a specific teacher, attributing this change to the teacher’s effectiveness. They found enormous variation in the change in scores of students of particular teachers, and published the names of some teachers – both the “best” and “worst”. Further, the paper announced that it will soon release the approximate rankings of all individual teachers in LA.
Will public accountability of individual teacher performance contribute to improve education quality in Los Angeles? Is this something that other education systems around the world struggling with finding options to raise teaching quality and student learning outcomes consider?