What’s new in education research? Impact evaluations and measurement – July round-up

Last week I shifted to a new role at the World Bank, so I’m thinking (even more) about education. I’ve written before about how much has been happening in education impact evaluation research in recent years. Two weeks ago I wrote up a conference on education systems research. Here, I want to bring together some of the interesting work that I’ve seen pop up in just the month of June. (Some of these came out earlier but came across my radar in June.)

What do teachers know?

• The Service Delivery Indicators examined teacher knowledge across 8 African countries (see Table 4 here).
• Last month, results of a content knowledge test for Grade 6 and Grade 10 teachers in the Philippines came out. Test items were drawn from the K-12 curriculum in “English, Filipino, Mathematics, and Science.” Grade 6 teachers were assessed in a randomly chosen subject; Grade 10 teachers were assessed in the subject they teach. As you can see in the figure below, the high school teachers didn’t do so well. The authors provide a reasonably detailed guide to how this was measured.

Source: World Bank 2016
 

• Going back a bit (but I just saw it), this study examines teacher knowledge in Cambodia; they also explain in detail how they test teacher mathematical knowledge, which could serve as a useful guide. They use a 30-question instrument: 6 from previous national grade 6 assessments, 12 from previous national grade 9 assessments, 6 from the international assessment TIMSS (grade 8), and 6 questions that require pedagogical content knowledge (seeing if the teacher can spot the mistake in a student’s mathematical thinking), The take away? “Teacher training center (TTC) trainers, trainees, and teachers in service have alarmingly low mathematics knowledge.”

How much do teachers matter?

• To date, the best evidence on the value added of teachers comes from the United States. Last month, Bau and Das presented teacher value added estimates from Pakistan, “among the first estimates of teacher value added and its correlates in a low income country,” finding that “teacher quality matters as much or more for student outcomes in these contexts as in OECD countries: moving a student from a teacher in the fifth percentile to the ninety-fifth percentile leads to a 0.64 standard deviation increase in test scores, relative to a 0.39-0.55 increase in the United States.”

Are teachers happy?

• A study of 4,000 education sector workers in 1,200 schools in Madagascar from Burón and Lassibille shows that – unsurprisingly – teachers are more satisfied when they are (a) better paid, (b) receive training, (c) have access to pedagogical materials, and (d), when they are civil servants (i.e., contract teachers are less satisfied). Strikingly, teachers are less satisfied in schools where the school director checks in on them and discusses student learning issues with them. If you believe teacher satisfaction matters (either inherently or as a bridge to student learning), then these results matter. (Some of this could potentially be selection: teachers with better attitudes end up in better jobs.)

Information in higher education

• There has been some recent promising research on providing information to high school students. Avitabile and de Hoyos found that providing information on the relative salaries of those who complete upper secondary (high school) versus those who complete lower secondary (middle school) increased student test scores, albeit not graduation rates.
• In a newish study by Bonilla et al., local Colombian college graduates gave high school students an overview of the relative returns to finishing college versus finishing high school. They also provided info on options for financing tertiary education. The intervention didn’t affect tertiary enrollment rates, but it did “increase the likelihood of enrolling in a top-10 college by almost 50% of the mean.”

Does the conditionality of cash transfers matter for education?

• New work from Akresh et al. says yes, based on a randomized trial varying conditionality in Burkina Faso: “Conditionality was linked to older children enrolling in school and attending regularly and younger children receiving preventive health check-ups.”
• The RCT also varied distribution of payment to mothers versus fathers: “Giving cash to mothers does not lead to significantly better child health or education outcomes.”

How big are the returns to education?

• Well, it depends on how you ask. In a survey experiment in Tanzania, Serneels et al. compare a short versus long questionnaire and find significant upward bias in Mincerian estimates of returns with the short instrument. (They find no difference in estimated returns depending on whether the respondent answers for herself or if a proxy respondent is used.) 

Can we improve early grade reading? 

• Patrinos characterizes an early grade reading impact evaluation (RCT) in Papua New Guinea: Teachers were trained in pedagogical knowledge and skills and received regular monitoring and support. Student reading improved. 

Do school systems matter?

• Woessman reviews evidence on “what explains the large international differences in student achievement”? He also does some basic education production function regressions. His take away? “The available evidence suggests that differences in expenditures and class size play a limited role in explaining cross-country achievement differences, but that differences in teacher quality and instruction time do matter. This suggests that what matters is not so much the amount of inputs that school systems are endowed with, but rather how they use them.

What about indigenous language education?

• A recent study from Hynsjö and Damon uses the Young Lives data in Peru to compare Indigenous children in Quechua-language schools versus similar children in Spanish-language schools, controlling for a range of individual and school characteristics. Three key findings: (1) “Indigenous children in Quechua schools score 0.429σ higher in mathematics than Indigenous children in Spanish schools.” (2) “There is no evidence that these effects are caused by quantitative or language achievement acquired prior to entering school.” And (3) “There is no evidence that students in Quechua schools have lower performance on Spanish tests than those in Spanish schools.” Of course there are selection issues at work here, but the authors conclude – after various checks – that they are more likely to observe a downward than an upward bias.

Health and education

• Glewwe et al.’s paper on the impact of providing eyeglasses on academic performance (RCT) in rural China is out: “Our preferred estimates, which exclude township pairs for which students in the control township were mistakenly provided eyeglasses, indicate that wearing eyeglasses for one academic year increased the average test scores of students with poor vision by 0.16 to 0.22 standard deviations, equivalent to 0.3 to 0.5 additional years of schooling.”
• Fixed effects estimates from Zhang and Xu suggest that a water treatment program in China in the 1980s “increased the completed grades of education of rural youth by 1.1 years” and “completely eliminated the gender gap in education in treated villages.”

New data!

• Using census data, Lee and Lee construct a dataset of educational enrollment ratios by gender since 1820 for 111 countries. Guess what? It’s going up!

Source: Lee and Lee (2016)