Why are there so many poor evaluations of ICT use in education?

This page in:

Olbers' paradox is sometimes easier to wrap your head around than the question of why there are so many poor evaluations of ICT use in education | image attribution at bottomDespite increasing attention to the impact of ICT on teaching and learning in various ways, the ICT/education field continues to be littered with examples of poor evaluation work.  A few of them arrive in my in-box every week.

There are many potential reasons advanced for the general poor quality of much of this work.  One is simple bias -- many evaluations are done and/or financed by groups greatly invested in the success of a particular initiative, and in such cases findings of positive impact are almost foregone conclusions.  Many (too many, some will argue) evaluations are restricted to gauging perceptions of impact, as opposed to actual impact. Some studies are dogged by sloppy science (poor methodologies, questionable data collection techniques), others attempt to extrapolate finds from carefully nurtured, hothouse flower pilot projects in ways that are rather dubious. (The list of potential explanations is long; we'll stop here for now.)

More fundamental to all of this is a belief by many that we still don't know how to evaluate the impact of technology use in schools.  People in this camp often paraphrase, unconsciously or not, the Solow computer paradox ("You can see the computer age everywhere but in the productivity statistics").  It may be, as an infoDev study argues, that "more useful analyses of the impact of ICT can only emerge when the methods used to measure achievement and outcomes are more closely related to the learning activities and processes promoted by the use of ICTs."  Fair enough.  But what advice do we have to share while we wait for this to occur?

Some recent studies of potential interest that you might have missed (all links to direct downloads of PDF files):

Note: The image used at the top of this blog post from Kmarinas86 via Wikimedia Commons is used according to the terms of its GNU Free Documentation License.


Michael Trucano

Visiting Fellow, Brookings, and Global Lead for Innovation in Education, World Bank

Join the Conversation

The content of this field is kept private and will not be shown publicly
Remaining characters: 1000