So, you are about to start field research in education. Whether you are planning a randomized control trial or a quasi-experiment, hopefully these tips may help!
Devote time and energy towards recruiting and training enumerators (your survey personnel). Someone once said that training enumerators is 95% of the battle in conducting good field research. I would argue that that would be dramatically underestimating its importance. The enthusiasm and perseverance of the enumerators makes or breaks all the hard work that has gone into designing the experiment. And so, in general, devoting at least a week to training them and letting them pilot the tool is essential. I find that reminding enumerators of the higher purpose behind the study really helps as well – in a small way, our shared work is helping improve literacy and numeracy outcomes for children across the world and that’s something that they should rightfully take pride in.
Usually, university students or recent graduates make excellent enumerators – they still have the passion to travel and learn, are more comfortable with technology, better able to cope with the punishing schedule and can trek the extra three hours to reach a randomly selected school in far-flung areas.
Take pilots seriously. Peter Behrens, one of the leading figures of modernist design, famously remarked that “less is more.” That advice may work for design but does not work for field research! Once your instrument is ready and the enumerators have been trained, you need to test your instrument in the field. This is your opportunity to test questions and get the flow of the assessment right. As a rule of thumb, you need at least two pilots (with time to amend the instrument and brief the enumerators on the changes between both pilots). Hold a debrief/ reflection workshop on each day of the pilot and get feedback from enumerators. Also, try and pilot the instruments within the same province (and this goes without saying but you shouldn’t pilot your instrument in a school that’s part of the evaluation).
Use tablets for data collection. The life expectancy of lead researchers has declined by at least a few years as they have tried to manage the slow and painful process of transferring paper based data into an electronic database. Data entry has to be done twice and the reconciling inconsistences is a nightmare. Also, paper surveys can get misplaced, damaged or stolen prior to data entry. Data entry may take weeks or months (I know that I’m still waiting on the data collected through some paper based forms in February 2017). There are no ways of ensuring that enumerators key in information for all necessary questions nor can you set field restrictions for questions to get a clean dataset.
Tablets change all of that. If your sample is large enough, tablets tend to be more cost effective and save you money in the long run. Data can be collected even when tablets are offline and can be uploaded in real time. After every upload cycle, you can go through the data and tell the team leaders if you notice any particular patterns that they need to watch out for.
Data is mostly clean by the time you get it and you can devote your time and energy to analysis. There are hundreds of good open-source survey platforms out there. For example, the World Bank’s very own Survey Solutions and Harvard Humanitarian Initiative’s Kobo Toolkit would be two that are intuitive and have good form builder interfaces for new users.
Use appropriate fonts. If your experiment involves a literacy assessment component, you need to ensure that you use the right font for the letters, frequently used words and reading passages. It may seem trivial but using an unfamiliar font which is not in the curriculum will lead to lots of blank states from confused children. In particular, letters like ‘a,’ ‘t,’ and ‘q’ appear very differently in Calibri or Arial in comparison to what children learn in school when they learn to write in cursive. Personally, Microsoft’s Chalkboard is my font of choice for literacy assessments.
“Constant vigilance”. The advice of Barty Crouch Jr. to Harry Potter’s fourth-year classmates at Hogwarts is just as valuable to researchers. If you want to conduct good field research, you need robust monitoring and evaluation throughout the data collection period – tablets will help as they collect GPS data and time stamps of when data was collected (this ensures that data hasn’t been entered at 3 AM in the enumerator’s house in Accra when the experiment is in the Upper East Region!). I’d still recommend surprise M&E visits. Do a weekly check in with your team leaders and send a thank you message to every one of the enumerators on your team each week.
Build good relationships with the Ministry of Education/provincial or district education offices. Ensuring the independence of your research is important but if you don’t have Ministry of Education (MoE) buy-in from the start, your results will lead to nothing. Confirm Ministry of Education buy-in from the beginning (and no, that doesn’t mean just a signed letter of approval from the Permanent Secretary – that is a must, in any case). Present the research design to key partners across the Ministry of Education, invite them to the training and the monitoring visits. A weekly summary to all the relevant partners, MoE and provincial departments of education is also good practice.
Reflect and learn. You are out in the field, travelling through crocodile infested waters on a canoe or on quad bikes through muddy terrain. You are meeting students and teachers from countries other than your own, hoping to do your bit towards improving education. Recognize that not many have the chance to share in their experiences. The cups of chai for the long hours of analysis and the endless STATA loops are for your future self – at the moment, reflect and learn. Aside from the big lessons, you will pick up at least a dozen games that you can play with a handful of stones – I can recommend some experts!
With reports from the World Bank team in Liberia, Save the Children, Rich Jones and Madhulika Khanna.
Find out more about World Bank Group Education on our website and on Twitter.
Join the Conversation