We all love good news. This simple fact of life explains a well known syndrome known as publication bias: studies with positive results are more likely to be published than those with negative results. But the syndrome goes beyond academic publications.
In education as well as in other areas of public policy, the pressure to show results (and to justify budgets) creates strong incentives to report on positive stories over and above those showing a lack of results. It is, indeed, easier and more pleasant to write about what works than about what doesn’t work.
A few months ago we launched a new note series, "Evidence to Policy," (or E2P for short) to present in non-technical language results from impact evaluation studies the World Bank has conducted of human development programs. From the start, I wanted to ensure that E2P remains a vehicle for evidence-based development policy and not a vehicle for intellectual bragging and biased reporting.
The fourth issue of E2P gave us the opportunity to demonstrate our commitment to this approach. In Can Computers Help Students Learn? we summarize the findings of a study by Felipe Barrera and Leigh Linden of a major effort in Colombia to provide computers to public school students as a means to improve their learning outcomes. This careful randomized control trial found that there was simply no measurable difference in student performance between those that got and those that didn’t get the computers.
There are many possible interpretations of the results, which the study authors discuss. It is clear that how the computers were used (who used them, how frequently, for what subjects, etc.) was a contributing factor to the lack of impact on learning outcomes. This highlights one positive message of the study: just dropping computers in schools, without the proper preparation and the right environment, is unlikely to work. (See EduTech's list of 10 worst practices of technology use in education for other prime examples of what not to do.)
New technologies are all around us, making possible many things that were simply impossible in the past. We don’t want to throw the technology baby away with the dirty water of no results. This study –and its negative results—plays an extremely important role in reminding us that achieving results is not as simple as we sometimes seem to believe.
Publicizing what doesn’t work is a fundamental part of any approach to evidence-based policy. Lack of results is a likely outcome of any innovation. We should remain open and even celebrate those that bring us the bad news as they are helping us stay honest.
Photo credit: Image of 'computer misuse' comes from Wikimedia Commons
Join the Conversation