impact

When is the rigorous impact evaluation of development projects a luxury, and when is it a necessity?

This is a question asked in a new paper examing the Millennium Villages Project (MVP), a high profile initiative that, according to its web site, offers a "bold, innovative model for helping rural African communities lift themselves out of extreme poverty".

In the words of one of the authors of When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages, "We show how easy it can be to get the wrong idea about the project’s impacts when careful, scientific impact evaluation methods are not used. And we detail how the impact evaluation could be done better, at low cost."  The paper underscores the importance of comparing trends identified within a project activity with those in comparator sites if one is to determine the actual impact of a specific project.  This sentiment should come as no surprise to those familiar with an area of exploding interest in the international donor and development community -- that of the usefulness of randomized evaluations.

In business and in international development circles, much is made about the potential for 'learning from best practice'.  Considerations of the use of educational technologies offer no exception to this impulse.  That said, 'best practice' in the education sector is often a rather elusive concept (at best!  some informed observers would say it is actually dangerous).  The term 'good practice' may be more useful, for in many (if not most) cases and places, learning from and adapting 'good' practices is often much more practical -- and more likely to lead to success.  Given that many initiatives seem immune to learning from either 'best' or even 'good' practice in other places or contexts, it may be most practical to recommend 'lots of practice', as there appears to be a natural learning curve that accompanies large scale adoption of ICTs in the education sector in many countries -- even if this means 'repeating the mistakes' of others.

But do we really need to repeat the mistakes of others? If adopting 'best practice' is fraught with difficulties, and 'good practice' often noted but ignored, perhaps it is useful instead to look at 'worst practice'.  The good news is that, in the area of ICT use in education, there appears to be a good deal of agreement about what this is!

If you have had your fill of theories and promises about what the widespread diffusion of information and communication technologies (ICTs) might mean for teaching and learning practices across an entire education system and want to see what actual practice looks like, a trip to Montevideo (or better yet, one of the regions outside the Uruguayan capital) should be high on your list.

Under Plan Ceibal (earlier blog post here), Uruguay is the first country in the world to ensure that all primary school students (or at least those in public schools) have their own personal laptop.  For free.  (The program is being extended to high schools, and, under a different financial scheme, to private schools as well).  Ceibal is about more than just 'free laptops for kids', however.  There is a complementary educational television channel. Schools serve as centers for free community wi-fi, and free connectivity has been introduced in hundreds of municipal centers around the country as well.  There are free local training programs for parents and community members on how to use the equipment.  Visiting Uruguay last week, I was struck by how many references there were to 'one laptop per teacher' (and not just 'one laptop per child', which has been the rallying cry for a larger international initiative and movement). Much digital content has been created, and digital learning content is something that is expected to have a much greater prominence within Ceibal now that the technology infrastructure is largely in place.

 It's been four years since the The One Laptop Per Child (OLPC) project (known then as the '$100 laptop) was announced.   According to recent unconfirmed news reports from India, one quarter million of the little green and white OLPC XO laptops are now on order for use in 1500 hundred schools on the subcontinent.  Four years on, what have we learned about the impact of various OLPC pilots that might be of relevance to a deployment in India?  Thankfully, preliminary results are starting to circulate among researchers.  While nothing yet has approached what many consider to be the gold standard of evaluation work in this area, some of this research is beginning to see the light of day (or at least the Internet) -- and more is planned.