227 studies later, what actually works to improve learning in developing countries?

|

This page in:

Yesterday we talked about some of the limitations in systematic reviews of educational research, and how many of the reviews have – on the face of them – varying recommendations. The main recommendations as to what works (principally drawn from the abstracts and introductions) are in the figure below.
 
 
Across these reviews of 227 studies, is there any overlap?
Despite what appear to be pretty disparate recommendations, we did find some overlap in our recent paper. This requires digging through the alternative classifications. Here are three kinds of interventions that are recommended across multiple reviews, together with some (not all) of the studies backing them up:

1. Pedagogical interventions that match teaching to individual student learning levels
  1. Assign students to separate classes based on initial ability so that teachers can focus instruction at the level of learning of individual students in Kenya (Duflo, Dupas & Kremer 2011) [4 reviews]
  2. Use math software to help students learn at their own pace (Banerjee et al. 2007) [5 reviews]
    • But just giving out laptops or desktop computers won’t guarantee the gains
  3. Train teachers to use an initial reading assessment and then continually assess student performance in Liberia (Piper & Korda 2011) [2 reviews]
As discussed before, there is lots of variation within categories. So getting the specifics right is crucial.  

2. Individualized, repeated teacher training, associated with a specific method or task
  1. Train teachers and provide them with regular mentoring to implement early grade reading instruction in local language in Kenya & Uganda (Lucas et al. 2014) [3 reviews]
  2. Provide local contract teachers with two weeks of initial training but reinforcement throughout the year in India (Banerjee et al. 2007) [5 reviews]
  3. Combine student reading groups with in-school supervisors to provide ongoing guidance to group leaders in Chile (Cabezas et al. 2012) [2 reviews]
  4. Help teachers learn to use storybooks and flash cards in India (He et al. 2009) [1 review]
    • As opposed to a similar (not identical) program introduced without teacher preparation (He et al. 2008) [3 reviews]
Note that none of these are what we often think of as teacher training (programs where you go to a training center for a few hours a month and listen to a lecture about pedagogy tend to be ineffective). Several simply help teachers to learn how to take advantage of some other intervention (in line with Mike Trucano’s point that technology isn’t replacing teachers).

3. Accountability-boosting interventions
  1. Provide teacher performance incentives in India (Muralidharan & Sundararaman 2011) [3 reviews]
    • Or even just provide teacher attendance incentives in India (Duflo et al. 2012) [4 reviews]
    • But it’s important to think about how best to design incentives so as to maximize learning while minimizing strategic responses to avoid what happened in Kenya (Glewwe et al. 2010) [5 reviews]
  2. Supplement civil service teachers with locally hired teachers on short term contracts in Kenya (Duflo, Dupas, & Kremer 2012) [4 reviews]
As some colleagues have pointed out, (1) and (2) have a common thread: Make sure that teaching, whether to teachers or to students, is adapted to the level of the recipient.

Are these the only ways to improve student learning based on current evidence? Of course not. There are many studies reported in the systematic reviews that had significant impacts despite not falling into any of these three categories. But this is where there is most overlap across the systematic reviews, so it’s not a bad place to start when thinking about what is likely to work. The reviews themselves have a considerably richer discussion of the nuances around these findings.

These conclusions are, of course, based on our own narrative review of these systematic reviews, so you may wish to examine the 227 underlying studies yourself and sum them up using your own mental weighting and subject to your own prior beliefs. Let us know what you find. Happy reading!
 
227 study titles in one figure
Topics

Authors

David Evans

Senior Fellow, Center for Global Development

Anna Popova

Researcher, Stanford University

David Evans
March 09, 2015

An anonymous commenter pointed out this quote from Duflo, Dupas, & Kremer (2011) - in point 1 above: "While tracking increases test scores for students at all levels of the pretest distribution assigned to be taught by contract teachers… initially low-scoring students did not benefit from tracking if assigned to a civil-service teacher”.

So ability tracking all by itself didn't have the positive benefit; it needed to be combined with the accountability intervention (point 3 above).

I also discovered another paper after writing this, by Duflo et al. (2015) [http://www.3ieimpact.org/media/filer_public/2015/02/24/ie_22_evaluation…] From a cursory glance, it seems that once again there are gains to a form of ability tracking: "students are given a brief assessment of basic Hindi skills at the start of the academic year, and a portion of the school day is set aside to group and teach students according to ability level, regardless of age or grade." Once again, this isn't plain ability tracking. It's a very particular format. But it IS a form of ability tracking, giving the teacher a chance to target teaching to the students' ability levels.