The majority of CCT programs with schooling conditions have been found to increase enrollment rates and attendance. Far fewer of the evaluations, however, report results on learning outcomes. Those that do typically find no gains in learning, at least as assessed by test scores. The 2009 CCT review report by Fiszbein, Schady, and others summarizes four studies that measure CCT impacts on learning outcomes. The first two use school-based testing data and find no impact on test scores. Since in-school test data are subject to various selection biases, two other studies rely on home testing of students but these studies also find no impact of the CCT program on learning outcomes.
My initial review of the more recent literature has found several papers that report the now expected gains in school enrollment or attendance, in a variety of settings, but only one example of CCT-induced gains in test scores – this from rural Malawi by Baird, McIntosh, and (yes, our co-blogger Berk) Ozler. A general finding that stands in stark contrast to cognitive measures in early childhood, where CCTs have been found to have an effect in Nicaragua and Ecuador. (A caveat – I am not nearly as familiar with this literature as some of my co-bloggers, but trying my best to get up to speed. If I’ve missed other examples of test score gains, please tell me!)
What might explain this disconnect between increased schooling yet no gain in test-assessed learning? There are three main reasons bandied about:
- Poor school quality. The mean quality of schools may be insufficient to actually affect the learning of students, or at least the learning of the marginal student induced to attend by the CCT program. And this selection certainly seems to be a factor - Filmer and Schady demonstrate that the children induced to stay in school by a Cambodian CCT program are drawn from relatively low levels of ability as assessed by baseline tests.
- Important non-school determinants of learning unaffected by the CCT. It is possible that the relative contribution of school attendance to the knowledge/skill production function is sufficiently small or partial. Hence the program-induced increase in attendance does not translate to higher test scores because other important factors such as home environment may be unaffected.
- Mis-measurement. The tests may be inappropriate or noisy estimates of skills or knowledge thus obscuring any true gain in learning. One contributing factor to the mis-measurement is that performance in part relies on student effort during the test – and during the in-home survey based testing that I’ve directly observed I’ve often wondered about the student effort put forth.
While the disconnect between enrollment and test scores has justifiably renewed interest in complimentary interventions to strengthen the supply side of education, all three of these factors can play concurrent important role and we shouldn’t discount the potential for learning even when children attend a poor-quality school.
I say this because there is a good deal of emerging evidence on the neuro-cognitive effects of education. Even learning basic literacy and numeracy under rudimentary conditions leads to observable cognitive enhancements – this is because the brain is the only human organ that exhibits profound structural change (plasticity) after birth. These brain effects of training can be quickly detected by MRIs and other brain scans – one study of medical students at 3 months before a major exam, right before the exam, and three months after showed expansion in the hippocampal region throughout the study period and an expansion in grey matter volume during the intensified study period, an expansion that persisted three months after the exam. (Grey matter is the neuronal mass in the brain primarily associated with processing and cognition.)
I should mention that the disconnect between enrollment and test-scores is not limited to CCT evaluations. A well-known study that prophylactically treated school children for malaria in Kenya found significant reductions in anemia, presumably due to a reduced incidence of malaria infection, and increases in cognitive scores. However the researchers found no significant gains in tests of learning and knowledge and speculate that poor quality schools account for this discrepancy. Undoubtedly the quality and condition of classroom instruction played a role. However I still wonder about the sensitivity of the learning tests –the test scores were indeed higher in the treatment group but this difference not precisely estimated.
So the three reasons listed above may coalesce to create a situation where there indeed are learning gains from school attendance, although the gains are slight due to poor school quality and the testing instrument not sensitive enough to pick up these gains. The assessment test is by construction an imperfect measure of the learning that takes place (since the limited length assures a lack of comprehensiveness) and, on top of that, test performance requires effort which is beyond the evaluator’s control. Of course brain MRIs aren’t feasible in a field-research setting (at least not yet - let’s check back in 5 years) so we are likely stuck with test based assessments of learning and skill formation. Developing more sensitive learning measurement tools will be a fruitful direction in evaluative research.
Join the Conversation