Syndicate content

impact

Let them eat laptops?*

Michael Trucano's picture

in my hand I have a very precious gift for youAs a result of reading the recent IDB study on the impact of the One Laptop Per Child project in Peru, my World Bank colleague Berk Ozler recently published a great post on the World Bank's Development Impact blog asking "One Laptop Per Child is not improving reading or math. But, are we learning enough from these evaluations?

Drawing insights from his readings of a few evaluations of technology use (one in Nepal [PDF] and one in Romania) he notes that, at quick glance, some large scale implementations of educational technologies are, for lack of a more technical term, rather a 'mess':

"The reason I call this a mess is because I am not sure (a) how the governments (and the organizations that help them) purchased a whole lot of these laptops to begin with and (b) why their evaluations have not been designed differently – to learn as much as we can from them on the potential of particular technologies in building human capital."

Three members of the team at IDB that led the OLPC Peru evaluation have responded ("One Laptop per Child revisited") in part to question (b) in the portion of Berk's informative and engaging post excerpted above.  I thought I'd try to try to help address question (a).

First let me say: I have no firsthand knowledge of the background to the OLPC Peru project specifically, nor of the motivations of various key actors instrumental in helping to decide to implement the program there as it was implemented, beyond what I have read about it online. (There is quite a lot written about this on the web; I won't attempt to summarize the many vibrant commentaries on this subject, but, for those who speak Spanish or who are handy with online translation tools, some time with your favorite search engine should unearth some related facts and a lot of opinions -- which I don't feel well-placed to evaluate in their specifics.) I have never worked in Peru, and have had only informal contact with some of the key people working on the project there.  The World Bank, while maintaining a regular dialogue with the Ministry of Education in Peru, was not to my knowledge involved in the OLPC project there in any substantive way. The World Bank itself is helping to evaluate a small OLPC pilot in Sri Lanka; a draft set of findings from that research is currently circulating and hopefully it will be released in the not too distant future.

That said, I *have* been involved in various capacities with *lots* of other large scale initiatives in other countries where lots of computers were purchased for use in schools and/or by students and/or teachers, and so I do feel I can offer some general comments based on this experience, in case it might of interest to anyone.

Ten things about computer use in schools that you don't want to hear (but I'll say them anyway)

Michael Trucano's picture

I don't want to hear thisAt an event last year in Uruguay for policymakers from around the world, a few experts who have worked in the field of technology use in education for a long time commented that there was, in their opinion and in contrast to their experiences even a few years ago, a surprising amount of consensus among the people gathered together on what was really important, what wasn't, and on ways to proceed (and not to proceed).  Over the past two years, I have increasingly made the same comment to myself when involved in similar discussions in other parts of the world.  At one level, this has been a welcome development.  People who work with the use of ICTs in education tend to be a highly connected bunch, and the diffusion of better (cheaper, faster) connectivity has helped to ensure that 'good practices and ideas' are shared with greater velocity than perhaps ever before.  Even some groups and people associated with the 'give kids computers, expect magic to happen' philosophy appear to have had some of their more extreme views tempered in recent years by the reality of actually trying to put this philosophy into practice.

That said, the fact that "everyone agrees about most everything" isn't always such a good thing.  Divergent opinions and voices are important, if only to help us reconsider why we believe what we believe. (They are also important because they might actually be right, of course, and all of the rest of us wrong, but that's another matter!) Even where there is an emerging consensus among leading thinkers and practitioners about what is critically important, this doesn't mean that what is actually being done reflects this consensus -- or indeed, that this consensus 'expert' opinion is relevant in all contexts.

Evaluating One Laptop Per Child (OLPC) in Peru

Michael Trucano's picture

learning learningFew would argue against the notion that the One Laptop Per Child project (OLPC, originally referred to by many as the '$100 laptop project') has been the most high profile educational technology initiative for developing countries over the past half-decade or so. It has garnered more media attention, and incited more passions (pro and con), than any other program of its kind. What was 'new' when OLPC was announced back in 2005 has become part of mainstream discussions in many places today (although it is perhaps interesting to note that, to some extent, the media attention around the Khan Academy is crowding into the space in the popular consciousness that OLPC used to occupy), and debates around its model have animated policymakers, educators, academics, and the general public in way that perhaps no other educational technology initiative has ever done. Given that there is no shortage of places to find information and debate about OLPC, this blog has discussed it only a few times, usually in the context of talking about Plan Ceibal in Uruguay, where the small green and white OLPC XO laptops are potent symbols of the ambitious program that has made that small South American country a destination for many around the world seeking insight into how to roll out so-called 1-to-1 computing initiatives in schools very quickly, and to see what the results of such ambition might be.

The largest OLPC program to date, however, has not been in Uruguay, but rather in Peru, and many OLPC supporters have argued that the true test of the OLPC approach is perhaps best studied there, given its greater fealty to the underlying pedagogical philosophies at the heart of OLPC and its focus on rural, less advantaged communities. Close to a million laptops are meant to have been distributed there to students to date (902,000 is the commonly reported figure, although I am not sure if this includes the tens of thousands of laptops that were destroyed in the recent fire at a Ministry of Education warehouse). What do we know about the impact of this ambitious program?

Scaling Social Impact in the North East with Ashoka Fellows

Parvathi Menon's picture

In Calcutta a few days before Christmas, December 2011, Ashoka India brought together Fellows from the North and North East around a thematic workshop with Innovation Alchemy. The theme was ‘Scale’. The issue of increasing the IMPACT of the work that the Fellows are implementing through their diverse initiatives.

The two days of engagement was a quick immersion into the complex Development world of the North East. The region is perceptibly isolated from the rest of the country, politically, geographically, economically... A brief research of the core challenges in this part of the country points to porous borders, leading to migration, infiltration and huge demand on a weak economy. High degree of ecological instability and recurring natural disasters repeatedly impacting livelihoods, increasing displacement and further reducing opportunities. Adding to the complexity is a feeling that ‘the Central Government does not care about the North East‘.

Combine all this – human rights struggles, cross-border violations, weak economy, limited opportunity and lack of any strong progressive policy frameworks – and what you get is a situation ripe for human conflict.

Surveying ICT use in education in Brazil

Michael Trucano's picture

Brazilian students queuing for their daily bread -- will their daily Internet be far behind?An on-going series in the New York Times ('Grading the digital school') is exploring the impact of educational technology programs in U.S. schools. One recent article in this series noted that "Hope and enthusiasm are soaring here. But not test scores." This phenomenon is not limited to schools in rich countries like the United States, of course:

"Although the government has invested resources in ensuring the broad use of ICT in education, the results of this use in meeting the goals and targets of educational programs are, however, virtually unknown."

This statement, which could apply to scores of countries around the world, can be found near the very start of TIC Educação 2010 ("ICT Education 2010"), a fascinating new survey on the use of ICTs in Brazilian schools.

Will Possible Labor Policies by Gulf Countries Affect Remittances to South Asia?

Ceren Ozer's picture

My entry last week gave a quick profile of the South Asian overseas workers and discussed the crucial role of remittances received from the Gulf Cooperation Council (GCC) countries (Saudi Arabia, the U.A.E, Kuwait, Qatar, Bahrain and Oman) for South Asian economies. Today I’d like to discuss whether changes in the labor market policies of the GCC countries could jeopardize job prospects for South Asian migrant workers.

Creating jobs for GCC citizens is already on the top of the agenda in some of these countries and is bound to gain more momentum with the youth bulge. Efforts to create jobs for nationals through the “nationalization of the labor market” have been further intensified as a response to the recent events in the Middle East. Across the GCC, additional policy measures are being announced highlighting the need to replace expats with nationals in private and public sector. These messages have been the strongest in Saudi Arabia, but also in the U.A.E. and Kuwait.

How Do You Measure History?

Anne-Katrin Arnold's picture

Over and over again, and then again, and then some more, we get asked about evidence for the role of public opinion for development. Where's the impact? How do we know that the public really plays a role? What's the evidence, and is the effect size significant? Go turn on the television. Go open your newspaper. Go to any news website. Do tell me how we're supposed to put that in numbers.

Here's a thought: maybe the role of public opinion in development is just too big to be measured in those economic units that we mostly use in development? How do you squeeze history into a regression model? Let's have a little fun with this question. Let's assume that
y = b0 + b1x1 + b2x2 + b3x3 + b4x4 + b5x5 + b6x6 + b7x7 + b8(x1x4) + b9(x3x4) + e

Learning from a randomized evaluation of OLPC in Peru

Michael Trucano's picture

some times the goals are clear to see -- it's just challenging to get there | image credit: Martin St-Amant - Wikipedia - CC-BY-SA-3.0The Inter-american Development Bank (IDB) recently released the first set of results from its on-going, multi-year randomized evaluation of the impact of the OLPC project in Peru.
Experimental Assessment of the Program "One Laptop Per Child" in Peru (Spanish version here) is the first rigorous attempt to examine the impact of the largest '1-to-1 computing' initiative in a developing country.  This evaluation, done in concert with the Ministry of Education, looks at the ambitious program to provide computing resources to multi-grade rural elementary schools in some of the poorer communities of Peru.

Evaluating the evaluating of the Millennium Villages Project

Michael Trucano's picture

not all millennium projects are this neatly contained within clearly defined bordersWhen is the rigorous impact evaluation of development projects a luxury, and when is it a necessity?

This is a question asked in a new paper examing the Millennium Villages Project (MVP), a high profile initiative that, according to its web site, offers a "bold, innovative model for helping rural African communities lift themselves out of extreme poverty".

In the words of one of the authors of When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages, "We show how easy it can be to get the wrong idea about the project’s impacts when careful, scientific impact evaluation methods are not used. And we detail how the impact evaluation could be done better, at low cost."  The paper underscores the importance of comparing trends identified within a project activity with those in comparator sites if one is to determine the actual impact of a specific project.  This sentiment should come as no surprise to those familiar with an area of exploding interest in the international donor and development community -- that of the usefulness of randomized evaluations.

How would you design an ICT/education program for impact?

Michael Trucano's picture

where is this road leading us? the path ahead is murky | image attribution at bottomImagine, if you will, that you were an official at an international development organization who has been working with country x for a number of years in helping them think through options and issues related to the use of ICTs in their education sector.  As part of this dialogue, you had regularly preached the virtues of a commitment to rigorous monitoring and impact evaluation.

Country x has, in various ways, been host to numerous initiatives to introduce computers into its schools and, to lesser extents, to train teachers and students on their use, and schools have piloted a variety of digital learning materials and education software applications.  It is now ready, country leaders say, to invest in a rigorous, randomized trial of an educational technology initiative as a prelude to a very ambitious, large-scale roll-out of the use of educational technologies nationwide. It asks:

What programs or specific interventions should we consider?


Pages