227 studies later, what actually works to improve learning in developing countries?
This page in:
Yesterday we talked about some of the limitations in systematic reviews of educational research, and how many of the reviews have – on the face of them – varying recommendations. The main recommendations as to what works (principally drawn from the abstracts and introductions) are in the figure below.
Across these reviews of 227 studies, is there any overlap?
Despite what appear to be pretty disparate recommendations, we did find some overlap in our recent paper. This requires digging through the alternative classifications. Here are three kinds of interventions that are recommended across multiple reviews, together with some (not all) of the studies backing them up:
1. Pedagogical interventions that match teaching to individual student learning levels
2. Individualized, repeated teacher training, associated with a specific method or task
3. Accountability-boosting interventions
Are these the only ways to improve student learning based on current evidence? Of course not. There are many studies reported in the systematic reviews that had significant impacts despite not falling into any of these three categories. But this is where there is most overlap across the systematic reviews, so it’s not a bad place to start when thinking about what is likely to work. The reviews themselves have a considerably richer discussion of the nuances around these findings.
These conclusions are, of course, based on our own narrative review of these systematic reviews, so you may wish to examine the 227 underlying studies yourself and sum them up using your own mental weighting and subject to your own prior beliefs. Let us know what you find. Happy reading!
Across these reviews of 227 studies, is there any overlap?
Despite what appear to be pretty disparate recommendations, we did find some overlap in our recent paper. This requires digging through the alternative classifications. Here are three kinds of interventions that are recommended across multiple reviews, together with some (not all) of the studies backing them up:
1. Pedagogical interventions that match teaching to individual student learning levels
- Assign students to separate classes based on initial ability so that teachers can focus instruction at the level of learning of individual students in Kenya (Duflo, Dupas & Kremer 2011) [4 reviews]
- Use math software to help students learn at their own pace (Banerjee et al. 2007) [5 reviews]
- Train teachers to use an initial reading assessment and then continually assess student performance in Liberia (Piper & Korda 2011) [2 reviews]
2. Individualized, repeated teacher training, associated with a specific method or task
- Train teachers and provide them with regular mentoring to implement early grade reading instruction in local language in Kenya & Uganda (Lucas et al. 2014) [3 reviews]
- Provide local contract teachers with two weeks of initial training but reinforcement throughout the year in India (Banerjee et al. 2007) [5 reviews]
- Combine student reading groups with in-school supervisors to provide ongoing guidance to group leaders in Chile (Cabezas et al. 2012) [2 reviews]
-
Help teachers learn to use storybooks and flash cards in India (He et al. 2009) [1 review]
- As opposed to a similar (not identical) program introduced without teacher preparation (He et al. 2008) [3 reviews]
3. Accountability-boosting interventions
-
Provide teacher performance incentives in India (Muralidharan & Sundararaman 2011) [3 reviews]
- Or even just provide teacher attendance incentives in India (Duflo et al. 2012) [4 reviews]
- But it’s important to think about how best to design incentives so as to maximize learning while minimizing strategic responses to avoid what happened in Kenya (Glewwe et al. 2010) [5 reviews]
-
Supplement civil service teachers with locally hired teachers on short term contracts in Kenya (Duflo, Dupas, & Kremer 2012) [4 reviews]
- Or in India (Banerjee et al. 2007) [5 reviews]
Are these the only ways to improve student learning based on current evidence? Of course not. There are many studies reported in the systematic reviews that had significant impacts despite not falling into any of these three categories. But this is where there is most overlap across the systematic reviews, so it’s not a bad place to start when thinking about what is likely to work. The reviews themselves have a considerably richer discussion of the nuances around these findings.
These conclusions are, of course, based on our own narrative review of these systematic reviews, so you may wish to examine the 227 underlying studies yourself and sum them up using your own mental weighting and subject to your own prior beliefs. Let us know what you find. Happy reading!
227 study titles in one figure
An anonymous commenter pointed out this quote from Duflo, Dupas, & Kremer (2011) - in point 1 above: "While tracking increases test scores for students at all levels of the pretest distribution assigned to be taught by contract teachers… initially low-scoring students did not benefit from tracking if assigned to a civil-service teacher”.
So ability tracking all by itself didn't have the positive benefit; it needed to be combined with the accountability intervention (point 3 above).
I also discovered another paper after writing this, by Duflo et al. (2015) [http://www.3ieimpact.org/media/filer_public/2015/02/24/ie_22_evaluation…] From a cursory glance, it seems that once again there are gains to a form of ability tracking: "students are given a brief assessment of basic Hindi skills at the start of the academic year, and a portion of the school day is set aside to group and teach students according to ability level, regardless of age or grade." Once again, this isn't plain ability tracking. It's a very particular format. But it IS a form of ability tracking, giving the teacher a chance to target teaching to the students' ability levels.