Collecting data in education can be a tricky business. After spending considerable resources to design a representative study, enlist and train data collectors, and organize the logistics of data collection, we want to ensure that we capture as true a picture of the situation on the ground as possible. This can be particularly challenging when we attempt to measure complex concepts, such as child development, learning outcomes, or the quality of an educational environment.
Data can be biased by many factors. For example, the very act of observation by itself can influence behavior. How can we expect a teacher to behave “normally” when outsiders sit in her or his classroom taking detailed notes about everything they do? Social desirability bias, where subjects seek to represent themselves in the most positive light, is another common challenge. Asking a teacher, “Do you hit children in your classroom?” may elicit an intense denial, even if the teacher still has a cane in one hand and the ear of a misbehaving child in another.
The first time a World Bank education team tried classroom observations in Brazil, it nearly provoked a state-wide teachers’ strike. It was October 2009 in the northeast state of Pernambuco and two members of the team, Barbara Bruns and Madalena Dos Santos, had handed out stopwatches to school supervisors newly trained in using the Stallings “classroom snapshot” method to measure teacher activities.
Two days later, the stopwatches were on the front page of Pernambuco’s leading newspaper: the teachers’ union called for a state-wide strike to protest an evaluation tool they dubbed the “Stalin method.”
“I thought the grant money we had used to train observers was down the drain,” recalled Bruns, a World Bank retiree now a visiting Fellow at the Center for Global Development. “But the governor, Eduardo Campos, was unfazed. He publicly declared: ‘No one is going to stop me and my secretariat from going into public schools to figure out how to make them better.’ The union backed down and the fieldwork went ahead.”
So, you are about to start field research in education. Whether you are planning a randomized control trial or a quasi-experiment, hopefully these tips may help!
Devote time and energy towards recruiting and training enumerators (your survey personnel). Someone once said that training enumerators is 95% of the battle in conducting good field research. I would argue that that would be dramatically underestimating its importance. The enthusiasm and perseverance of the enumerators makes or breaks all the hard work that has gone into designing the experiment. And so, in general, devoting at least a week to training them and letting them pilot the tool is essential. I find that reminding enumerators of the higher purpose behind the study really helps as well – in a small way, our shared work is helping improve literacy and numeracy outcomes for children across the world and that’s something that they should rightfully take pride in.
Statistics. Either you love or hate them. We certainly need them to compare and measure data, as well as to make informed decisions. Here at the World Bank, we often get calls from researchers, students and journalists asking for education data: Is there an increase in the number of tertiary education students in Brazil in 2017? How much are governments in South Asia spending on education? Where can we find a database of World Bank education projects?
We try to help answer these, as much as we can, but a quicker and easier way of finding this data is to visit the World Bank’s revamped EdStats website. EdStats – the World Bank’s portal for accessing education-related data – has been around since 1998 and is one of the most used websites by education specialists at the World Bank and partner organizations. User feedback has been highly positive: the interface looks neater, highly mobile and tablet-friendly. Allow me to give you a “tour” of the revamped website.
See if you can spot the pattern:
- “Although the quantity of schooling has expanded rapidly, quality is often abysmal.” (Kremer et al.)
- “Between 1999 and 2009, an extra 52 million children enrolled in primary school…Yet the quality of education in many schools is unacceptably poor.” (Krishnaratne et al.)
- “Progress over the last decade in regards to school access and enrollment has been promising.” But “current learning levels for primary as well as secondary school students are extremely low in much of Sub-Saharan Africa” (Conn)
- “The most consistent focus of investment has been on increasing primary and secondary school enrollment rates… More recently, however, attention has begun to swing toward the quality of schools and the achievement of students—and here the evidence on outcomes is decidedly more mixed.” (Glewwe et al.)
- “Over the past decade, low- and middle-income countries have made considerable progress in increasing the number of children and youth who enroll in school and stay long enough to learn basic skills… Learning in many low- and middle-income countries remains appallingly low.” (Murnane & Ganimian)
Again and again, we hear the refrain: access is improving, but learning lags. Thankfully, an increasing number of studies reveal interventions that work – and those that don’t – to improve learning around the world.
Every year, the World Bank generates a wealth of useful information about education systems across the globe, from project-driven appraisal documents and results stories to country-specific data and news to impact evaluations and everything in between. Through the Smarter Education Systems tool, this information, which can often be overwhelming to navigate and curate, is becoming more easily accessible, digestible, and searchable. The Smarter Education Systems tool demonstrates how the World Bank helps countries ensure "Learning for All" through support to countries on both the financing (loans, grants, and more) and knowledge (research, publications, and more) fronts.
Education and training play an important role in ensuring that youth develop the skills they need to live independent and prosperous lives. The research is clear: youth are more affected by unemployment than any other age group. Around the globe we have seen the political, economic and social consequences of young people not having jobs. Governments and international development organizations have turned to education and training initiatives as one tool to enable youth to find jobs or launch their own businesses.
Are we effective in presenting education data to help tackle the real issues that developing countries are facing? The education community continues to be puzzled by two realities: (1) crucial data is often not available and (2) available data is often hard to digest.
The new buzz words in the World Bank are Open Data. Here, in our blog, we have been championing the cause of Open Data (see New Open Data Initiative Emphasizes Importance of Education Stats) and what it does for knowledge sharing and looking at development solutions for Education systems.
You may know that the President Bob Zoellick (also known as RBZ) recently delivered a pretty inspiring speech at Georgetown University at the end of September. He was advocating for a new perspective for the Bank: “Beyond the Ivory Tower to a New Research Model: Open Data, Open Knowledge, Open Solutions.”