A few weeks ago, I participated in a debate on whether school grants buy student learning. I agreed to argue a firm no – as in, “No, school grants do NOT buy student learning” – because I believe the evidence to date strongly supports that. Here I’ll review that evidence.
Let me be clear about what I’m not saying.
Why school grants might work
Intuitively, it seems like maybe school grants could buy learning. For example, local communities may have the best information about what’s going wrong with their schools, so why not give them the resources to fix them rather than relying on some bureaucrat in Nairobi or Brasilia? In addition, communities may be more responsive to parents than the central government would be: An angry mother can go find an absent teacher and say, “Hey, go to school!” whereas it’s much harder for her to demand accountability from a distant education officer in the capital. (Beasley & Huillery have a nice discussion of these mechanisms.) This all seems sensible. And if we wanted to build our policies on intuition, we’d be all set.
But wait…
Let’s look at how learning works. Anna Popova and I recently reviewed 6 systematic reviews in education – covering 230 underlying evaluations – and what consistently improves student test scores? Pedagogy! Teachers knowing how to teach at the level of the student. Training teachers effectively – not just any teacher training – but training them with specific tools and repeated visits by true experts. (We also find some evidence for accountability boosting interventions, but these are driven by studies of teacher incentives and contract teachers, not flexible grants.)
Now, if that’s what works most consistently to improve student learning, would we expect that local communities would have any expertise in providing it? No. So frankly, I wouldn’t expect school grants to improve learning.
The evidence from reviews
But let’s look at the empirics: Michael Kremer and co-authors wrote in Science in 2013 that “test scores are…unresponsive to more-of-the-same inputs, such as…providing flexible grants.” Patrick McEwan wrote in the Review of Educational Research in 2015 that when it comes to learning, “Monetary grants…had mean effect sizes that were close to zero and not statistically significant.” (The 3ie review is more positive, but it looks like the only flexible school grants program it includes is the Mexico program I mentioned at the beginning of the post.)
And the evidence from individual studies
It’s true that sometimes we can find a positive impact in a grant program for some group somewhere. A school grant program in the Gambia increased test scores when combined with school management training and IF (and only if) you happen to be in a community that already had high levels of literacy. School grant programs in India and Zambia (ungated version) only increased student test scores when the grants were unexpected. When communities expected the grants, households lowered their own educational investments and the results disappeared. A school grant program in Senegal improved test scores for girls in 3rd grade one year after the program began. No impact on boys. No impact on girls in 5th grade. And even that initial impact on girls in 3rd grade disappeared by the following school year. Do grants improve learning? Sure, for 3rd grade girls, for a little while.
In Indonesia, grants by themselves had no impact. Grants with school committee training had no impact. Grants with school committee training PLUS an intervention linking the school to the village council had no impact. But grants with the link to the village council but NO school committee training did have an impact. If that’s confusing to you, you’re not alone. To use the rhetoric I adopted in the debate, it’s like a game of Twister. Do grants improve learning? Not on average, but maybe if you put your LEFT HAND on RED and your RIGHT FOOT on the TAIL of a PURPLE UNICORN.
Grants take time?
But maybe it takes time to get results from a school grant program. In Brazil, the government implemented a range of school management interventions, one of which was a school grant program. A decade later, one of Brazil’s champions of education research, Ricardo Paes de Barros, wrote that “educational performance tends to be better … where any of the three management innovations … had been adopted. The only exception is that school performance is lower…where transfer of resources to schools [that’s school grants] had been instituted.”
An exception
Now, as mentioned early on, I did find an exception in Mexico. There, the Support to School Management (or AGE) program financed parent associations and subsequently reduced grade failure by 7%, which is no mean feat. Unfortunately, as the authors wrote, the program “had no effect in extremely poor communities” and then – in a later conference abstract – they wrote that the program improved test scores on average but “we do not find significant results for females.”
Are these just bad grant programs?
You might argue that these aren’t very good grant programs. Nobody set out to put in place a mediocre grant program in Zambia or Indonesia or the Gambia. In several cases, the grant program that was put into place was not the grant program that was initially planned. That tells us that these are the grant programs that are in fact feasible in most of the countries we work with. So if what we’re arguing is that some idealized grant program could improve outcomes, then I can’t disprove that hypothetical. But I would argue that we should be interested in the best possible programs can be implemented in resource-poor environments.
Take away
Can school grants buy student learning? Maybe. For some students, in some sub-sets of schools, in some countries. But if we want to improve learning for a wide range of children, including the extremely poor, let’s focus on the programs that most consistently achieve that and let school grants accomplish other things (like increasing access to school).
Let me be clear about what I’m not saying.
- I’m not saying that school grants shouldn’t be part of an education program. We have evidence from Haiti, for example, that school grants can increase student enrollment. This may have value aside from learning outcomes. We have evidence from Mexico (ungated) that school grants can increase community involvement in the school, which – as came out in the debate – may be the first in many steps toward improving learning.
- I’m also not saying that there isn’t a single example in which school grants improve student test scores. An abstract on Mexico suggests an increase in student learning. There is also some evidence from rich countries, like Korea (ungated).
Why school grants might work
Intuitively, it seems like maybe school grants could buy learning. For example, local communities may have the best information about what’s going wrong with their schools, so why not give them the resources to fix them rather than relying on some bureaucrat in Nairobi or Brasilia? In addition, communities may be more responsive to parents than the central government would be: An angry mother can go find an absent teacher and say, “Hey, go to school!” whereas it’s much harder for her to demand accountability from a distant education officer in the capital. (Beasley & Huillery have a nice discussion of these mechanisms.) This all seems sensible. And if we wanted to build our policies on intuition, we’d be all set.
But wait…
Let’s look at how learning works. Anna Popova and I recently reviewed 6 systematic reviews in education – covering 230 underlying evaluations – and what consistently improves student test scores? Pedagogy! Teachers knowing how to teach at the level of the student. Training teachers effectively – not just any teacher training – but training them with specific tools and repeated visits by true experts. (We also find some evidence for accountability boosting interventions, but these are driven by studies of teacher incentives and contract teachers, not flexible grants.)
Now, if that’s what works most consistently to improve student learning, would we expect that local communities would have any expertise in providing it? No. So frankly, I wouldn’t expect school grants to improve learning.
The evidence from reviews
But let’s look at the empirics: Michael Kremer and co-authors wrote in Science in 2013 that “test scores are…unresponsive to more-of-the-same inputs, such as…providing flexible grants.” Patrick McEwan wrote in the Review of Educational Research in 2015 that when it comes to learning, “Monetary grants…had mean effect sizes that were close to zero and not statistically significant.” (The 3ie review is more positive, but it looks like the only flexible school grants program it includes is the Mexico program I mentioned at the beginning of the post.)
And the evidence from individual studies
It’s true that sometimes we can find a positive impact in a grant program for some group somewhere. A school grant program in the Gambia increased test scores when combined with school management training and IF (and only if) you happen to be in a community that already had high levels of literacy. School grant programs in India and Zambia (ungated version) only increased student test scores when the grants were unexpected. When communities expected the grants, households lowered their own educational investments and the results disappeared. A school grant program in Senegal improved test scores for girls in 3rd grade one year after the program began. No impact on boys. No impact on girls in 5th grade. And even that initial impact on girls in 3rd grade disappeared by the following school year. Do grants improve learning? Sure, for 3rd grade girls, for a little while.
In Indonesia, grants by themselves had no impact. Grants with school committee training had no impact. Grants with school committee training PLUS an intervention linking the school to the village council had no impact. But grants with the link to the village council but NO school committee training did have an impact. If that’s confusing to you, you’re not alone. To use the rhetoric I adopted in the debate, it’s like a game of Twister. Do grants improve learning? Not on average, but maybe if you put your LEFT HAND on RED and your RIGHT FOOT on the TAIL of a PURPLE UNICORN.
Grants take time?
But maybe it takes time to get results from a school grant program. In Brazil, the government implemented a range of school management interventions, one of which was a school grant program. A decade later, one of Brazil’s champions of education research, Ricardo Paes de Barros, wrote that “educational performance tends to be better … where any of the three management innovations … had been adopted. The only exception is that school performance is lower…where transfer of resources to schools [that’s school grants] had been instituted.”
An exception
Now, as mentioned early on, I did find an exception in Mexico. There, the Support to School Management (or AGE) program financed parent associations and subsequently reduced grade failure by 7%, which is no mean feat. Unfortunately, as the authors wrote, the program “had no effect in extremely poor communities” and then – in a later conference abstract – they wrote that the program improved test scores on average but “we do not find significant results for females.”
Are these just bad grant programs?
You might argue that these aren’t very good grant programs. Nobody set out to put in place a mediocre grant program in Zambia or Indonesia or the Gambia. In several cases, the grant program that was put into place was not the grant program that was initially planned. That tells us that these are the grant programs that are in fact feasible in most of the countries we work with. So if what we’re arguing is that some idealized grant program could improve outcomes, then I can’t disprove that hypothetical. But I would argue that we should be interested in the best possible programs can be implemented in resource-poor environments.
Take away
Can school grants buy student learning? Maybe. For some students, in some sub-sets of schools, in some countries. But if we want to improve learning for a wide range of children, including the extremely poor, let’s focus on the programs that most consistently achieve that and let school grants accomplish other things (like increasing access to school).
Join the Conversation