What’s new in education research? Impact evaluations and measurement – October round-up

Here is a curated round-up of recent research on education in low- and middle-income countries, with a few findings from high-income countries that I found relevant. All are from the course of 2016.

If I’m missing recent articles that you’ve found useful, please add them in the comments!

Teachers and teaching

• A guided instruction intervention in Chile (Bassi et al.), implemented in under-performing schools (and tested via RCT), mostly benefit only the best students. Also, the CLASS (classroom observation instrument) shows positive correlations between the quality of teacher-student interactions and the performance of low-income students. (My slightly longer take on this paper.)
• Low-performing teachers in the U.S. who are paired with high-performing teachers at the same school and invited to “focus on specific skills identified in the low-performer’s prior performance evaluations” improved their students’ test scores, and the effect persisted in the year after the intervention (Papay et al.)
• A series of experiments seeking to help teachers adapt instruction to the level of the student in India – and to integrate the programs into government education – was generally successful and extremely informative (Banerjee et al.). (My longer take on this paper.)
• Some of Banerjee et al.’s experiments involve separating students by ability (sometimes called “tracking”). In the U.S., creating a class for high achievers led to “significant effects that are concentrated among black and Hispanic participants. Minorities gain 0.5 standard deviation units in fourth-grade reading and math scores, with persistent gains through sixth grade.” No evidence of negative spillovers for kids in the regular classes (Card & Giuliano).
• Teacher expectations of students’ future performance seem to have a causal impact, even beyond their simple predictive power (Papageorge et al.).

• Incentives for student school attendance in Indian slums increased attendance. But when the incentives were removed, students who had low pre-incentive attendance saw both attendance and test scores fall. “For these students, the incentive was also associated with lower interest in school material and lower optimism and confidence about their ability” (Visaria et al.).
• This is US-based, but just saw a new addition to the student incentives literature (Levitt et al.): Modest, significant impacts on average, but larger effects for students near the threshold for the bonus. It didn’t matter whether the bonus went to the parents or the student.
• Conditions aren’t always enough: Evidence from cash transfers in Mexico suggest that “half of recipients forgo income for which they are eligible by failing to send children to school.” Who, specifically? “The poorest households, those with more dependents and high school students, recipients with limited education, and those living in large urban areas” (Heracleous et al.)
• Mbiti argues that many of the factors driving low learning levels in the developing world “reflect the low levels of accountability across multiple levels of the education system.” He reviews accountability interventions and looks forward.
• Accountability is, of course, more than just financial bonuses. Gill et al. have a useful paper that lays out the array of accountability mechanisms.
• A program that mailed summer reading books weekly in the U.S. and offered in-kind incentives to some students to read the books. “Students with greater motivation to read at baseline read more books in response to the incentives.” So the incentives worked most on students who needed them least. Reading comprehension outcomes were mixed (Guryan et al.).
• One potential incentive to enroll in primary is whether there is a secondary school available for you to attend. Mukhopadhyay & Sahoo find that a “1 km reduction in distance to secondary school increases probability of primary enrollment by 0.074” in India.
• Exploiting variation across multiple tests for the same students in Israel, a high pollution day leads to lower scores on high-stakes tests, which translates into measurable impacts on earnings later. “Reliance on noisy signals of student quality can lead to allocative inefficiency” (Ebenstein & Lavy).

• A survey on technology and education finds that “studies of ICT and CAI in schools produce mixed evidence with a pattern of null results” (Bulman & Fairlie). They are more optimistic, however, on the limited evidence from low- and middle-income countries, and on math-based computer-assisted learning programs.
• Can technology make it easier to scale coaching interventions to help college students? An experiment in Canada says yes and no: A one-time on-line exercise didn’t work and neither did advice and motivation via text messages. But matching students with “upper-year undergraduate coaches” led to a 0.3 s.d. increase in grades (Oreopoulos & Petronijevic).
• How can education technology help refugees? This World Bank report documents experiences in the Middle East and North Africa (Annex 1), brings together different advice on design principles for ICT in education (Annex 2). And here’s a call to evaluate: “The current situation provides numerous opportunities to build the evidence base, even carry out randomized control trials, and thereby improve ICT interventions and bring them to scale.” Someone get on that, please.
• Textbook spending in the U.S. led to significant (but small) positive test score effects at the primary level (Holden). No significant effects at middle or high school, but big standard errors.

• An intervention to improve reading in mother tongue in Kenya (in more than 150 schools) had positive literacy impacts, but also faced challenges: “Many educators were not speakers of the languages, some communities resisted mother tongue instruction, and some areas were more language heterogeneous” (Piper et al.).
• Mother-tongue instruction in Ethiopia raised enrollment in primary school and decreased overage-for-grade (Seid).

• Jackson finds that single-sex schools in Trinidad & Tobago leads to better test scores for both boys and girls and fewer arrests for boys.
• Does attending a school with higher performing peers positive impact performance? Evidence from China suggests no (Hoekstra et al.). At the very top high schools, there is a performance effect, but it seems to be driven by the quality of teachers.

• Who’d have thought? Ethnic groups that practice bride price in Indonesia and Zambia invest more in female education since it yields a higher bride price (Ashraf et al.).
• Heath & Jayachandran review trends in female education as well as labor force participation in developing countries. “There is considerable evidence of outcomes that both benefit women and achieve policy relevant goals. In particular, both education and labor force participation have been shown to delay fertility and lead to healthier children once a woman does have children.”
• What is the impact of terrorism on school enrollment, particularly for girls? Difference-in-differences analysis from Pakistan suggests “that low levels of exposure to terrorism had little effect on school enrolment. High levels of exposure reduced the enrolment rate for boys by about 5.5 percent and girls by about 10.5 percent” (Khan & Seltzer).
• In both Chile and Nicaragua, young girls (7 months to 6 years) score better on both tests of language and socioemotional skills than young boys (Bando et al.).
• Bharadwaj et al. examine the gender gap in mathematics in Chile: “We find that students have different perceptions (on the basis of gender) regarding their own math abilities. Even conditional on math score, we find that boys tend to be more optimistic about their math ability.”
• In Monrovia, Liberia, a survey shows that 95% of 12-15 year-old girls are in school, but the vast majority are overage for their grade (the median by 4 years). Of those who have dropped out, lack of money for school fees is the main reason (Koroknay-Palicz).

• A new scale for measuring early child development – the Engle Scale – is piloted in 4 Latin American countries (Verdisco et al.)
• Popova, Arancibia, and I have a new instrument out for characterizing teacher professional development (in-service teacher training programs) – here are the paper with the instrument and a summary blog post.
• How do different classroom observations instruments compare in Chile? Bruns et al. compare the Stallings and the CLASS.

• In Peru, “school administrative data provide novel evidence that reducing iron deficiency results in a large and significant improvement in school performance and aspirations for anemic students” (Chong et al.). The intervention involved making iron pills available at a health center and then using media messages to encourage teens to take them.

• A financial education program in 900 schools in Brazil improved knowledge of financial management, but outcomes on actual behaviors were mixed, with more saving but also more use of expensive credit (Bruhn et al.). Maybe kids thought: I didn’t realize I could use credit. What an amazing idea?

• “A recent study of student learning at three of Rwanda’s most prestigious public universities has suggested that Rwandan students are not improving in their critical thinking ability during their time at university.” Why not? A series of case studies reveal two main reasons: “Limited understanding of the rationale for pedagogical change and low levels of faculty motivation to implement more labor-intensive teaching methods” (Schendel).

• “We estimate that exposure to women's suffrage during childhood leads to large increases in educational attainment for children from economically disadvantaged backgrounds… An investigation into the mechanisms behind these effects suggests that the educational gains are plausibly driven by the rise in public expenditures following suffrage” (Kose et al.).

• If you want even more education links, see my July round-up.