Learning more with every year? Estimating the productivity of schooling in developing countries: Guest post by Abhijeet Singh

This is the fourteenth in our series of job market posts this year. 

Despite massive increases in school enrolment in developing countries, learning levels have lagged behind. But the range in average student achievement is large: In the 2012 PISA assessment (of 15-year-olds), Vietnamese students got higher scores than those in the US and the UK, but Peru ranked last (OECD 2012). The magnitude of the gap between these two developing countries was 1.4 standard deviations (SD); for comparison the difference between the US and Finland was 0.38SD.
 
My job market paper answers the question of how much of this gap reflects differences in the productivity of the schooling systems, as opposed to other factors such as nutrition, early childhood shocks, or endowments – a critical policy question relevant to the substantial education spending around the world.

Achievement gaps between Peru and Vietnam are modest at school entry and grow rapidly
 
Using unique panel data with comparable tests, I first establish that there is little difference between the ability of Peruvian and Vietnamese students before they enter school, at age 5. By the time these students reach 8 years old (in grade 2 or 3), a massive gap of 0.75 SD opens up.
 
This gap applies across the entire distribution of students, as the figure below illustrates. The left panel presents a mapping between percentiles of the Peruvian and Vietnamese distributions at 5 and 8 years of age --- for all percentiles of the Peru distribution, their relative standing to Vietnam is worse at 8 than at 5, indicating substantial divergence. The right panel looks at the dynamics of achievement, plotting the relationship between scores at age 5 and scores at age 8. Across the distribution of achievement, Vietnamese students do much better at 8 than Peruvian students with identical initial scores.




This finding, that gaps prior to school entry are modest at best but grow rapidly in the first couple of grades, is novel and highlights the critical importance of this period of education. So far, we just haven’t had the data to study learning dynamics across countries, especially at young ages, in a panel setting.
 
Estimating the productivity of schooling
 
That students start out the same at age 5 but diverge by age 8 seems to rule out pre-existing gaps as the cause of different ability later on. But can we really pin divergence down to the effectiveness of schooling systems? Maybe all this illustrates is that Vietnamese students learn faster over time regardless of school quality. To address this I estimate the causal effect of an additional year of schooling, using two approaches: panel-based “value-added” models (which have previously been validated against experimental and quasi-experimental estimates, for example these papers for the US, or my previous work in India) and a regression discontinuity based on age and school enrolment rules (see Figure below for the first stage).



A school year is much more productive in Vietnam – and this explains the divergence
 
Both the panel VA models and the regression discontinuity estimates tell the same story: the causal effect of a year of schooling in Vietnam (~0.78 SD) is much greater than in Peru (~0.4 SD). Accounting for the exposure to schooling and its differential productivity explains the full extent of the Vietnamese advantage in test scores. In concrete terms, Vietnamese students learned a similar amount per year at the simplest tasks (e.g., addition and subtraction) but much more in terms of applying these to more complex tasks (e.g., multiplication, division and applied problems).
 
What could explain this difference in the effect of a school year?
 
This estimate of schooling productivity – how much a student learns per year of schooling – is a system-level parameter. Akin to a TFP parameter, it reflects the composite effect of, for example, the effort of teachers, students and parents, instructional quality and curricula or school management.
 
Whilst I cannot directly investigate the sources of productivity differences with this data, I can compare features of the education systems (Section 5 in the paper). Overall there is little difference across the two samples in spending per student (PPP $), pupil-teacher ratios, the proportion of qualified teachers or total hours spent on academic activities per day.
 
Where the systems do seem to differ drastically is in accountability mechanisms. In Peru, de facto accountability is poor; salary, promotions and transfers are not based on performance; teachers are often absent at the time of surprise inspections; they spend less time-on-task in the classroom than expected; teacher feedback is infrequent and sometimes incorrect; and the cognitive demand of tasks is low. In contrast, these concerns seem to be much less prevalent in Vietnam, which – for example – runs an extensive teacher recognition scheme relying on peer evaluations of teachers based on classroom evaluations right from communes to the national level. It is plausible that accountability mechanisms could explain a portion of the productivity differences – an area worth further investigation.
 
A final caveat relates to “culture”. My results rule out that Vietnamese students were all born smarter or that, independent of schooling, are quicker learners. Cultural differences that interact with schooling, e.g., relating to student motivation when enrolled, may however still matter. But, to translate into test scores, these differences must be mediated through, for example, student and teacher effort or how teaching is organized. The intervention literature on education has shown that these behaviours are malleable (see for example these papers on incentives to students and teachers). So, little suggests that test score differences are immutable or that cultural differences, even if they are part of the story, constrain the role for policy.
 
What can we take away from this?

1. Typical developing countries will need to nearly double their school-year productivity in order to catch up to OECD levels of achievement: The Peruvian system is probably similar to many developing countries. The near-doubling of productivity that will be required for Peruvian students to catch up with the Vietnamese system (which is at OECD levels of achievement) illustrates the magnitude of the challenge that flailing education systems face.
2. The first years of schooling are a critical period of divergence: At least for these two countries, the test score divergence in primary schooling is substantial. This matters for policy, but also for research: even if PISA were to go to more developing countries, divergence happens much before their testing age of 15 years. Understanding this carefully will require data at earlier ages.
3. We need more comparable measurements of test scores and of school level processes: Generating comparable test measures is something I have pleaded for on this blog before. This is feasible and could help answer substantive questions that we just haven’t been able to yet.  This paper illustrates these analytical possibilities. For understanding why productivity differs across countries, we will also need comparable measures of school-level processes, tools for which are also increasingly available (e.g. the World Management Surveys or the Stallings classroom snapshots).  

The contribution of this paper is to apply microeconometric tools to comparably estimate the productivity of schooling systems and, substantively, to lay out the magnitude of the policy challenge – doubling the productivity of schooling. This is an ambitious goal. But recent experimental work does find effect sizes of this magnitude (including my recent paper with Karthik Muralidharan and Alejandro Ganimian). The key challenge is to take these reforms to scale (see Dave Evans’s recent blog post) which, if achieved, could be substantively important for growth and labour productivity.
 
Abhijeet Singh received his doctorate from the University of Oxford in 2015 and is a post-doc at University College London. More details about his research can be found on his personal website .