World Bank Blogs
Syndicate content

evaluation

To Index or Not To Index

Shanthi Kalathil's picture

Here’s an interesting twist on advocacy around anti-corruption: Global Integrity, which publishes the Global Integrity Report on governance and anti-corruption in 107 countries around the world, has stopped publishing its Global Integrity Index, which ranks countries according to their overall scores. While the report still contains quantitative data and qualitative reporting on the health of individual countries’ anti-corruption frameworks, the organization made a conscious decision to discontinue the index aspect of the report.

Why? Apparently, Global Integrity found that while the index generated good publicity for Global Integrity, it was less effective as an advocacy tool. (It also notes that it has scaled down the number of countries it covers, which gives the index less utility.) “Indices rarely change things,” notes Nathaniel Heller on the Global Integrity blog. “Country rankings are too blunt and generalized to be ‘actionable’ and inform real rebate and policy choices.”

Millennium Villages Project continues to systematically overstate its effects

Gabriel Demombynes's picture

by Michael Clemens and Gabriel Demombynes

The Millennium Villages Project (MVP) is an experimental anti-poverty interventionin villages across Africa. In October, we released evidence that the Project’s official publications were overstating its real effects, and we offered suggestions on improving its impact evaluation. On Tuesday the MVP, whose leadership and staff are aware of our work, continued to greatly overstate its impact.

What Are the Costs of Not Investing in ICTs in Education?

Michael Trucano's picture

empty pockets?Kentaro Toyama has started 2011 off 'with a bang' on our sister Education Technology Debate site, which is sponsored by our friends at infoDev and UNESCO. 

There is much to comment on in Kentaro's post, 'There Are No Technology Shortcuts to Good Education' -- to say nothing of the insights and assertions in the 100+ comments that follow it, many of them from people who are quite well known in the field.  Subsequent contributions on the ETD site from Larry Cuban, Cristobal Cobo, Claudia Urrea and Lowell Monke should provide further grist for debate and discussion.

Kentaro lays out a number of arguments in his piece.  One of them is the following:

"I’ve so far argued that technology in education has a poor historical record; that computers in schools typically fail to have positive impact (with the rare exceptions occurring only in the context of competent, well-funded schools); that information technology is almost never worth its opportunity cost; and that quality education doesn’t require information technology."

 
My aim here is not to contest (or support) any of the assertions in Kentaro's piece (I'd recommend you look in the comments section of the ETD site for this sort of thing).  Rather, it is to note that, in many instances, Kentaro's assumptions about what drives policy may well be beside the point.

Learning from a randomized evaluation of OLPC in Peru

Michael Trucano's picture

some times the goals are clear to see -- it's just challenging to get there | image credit: Martin St-Amant - Wikipedia - CC-BY-SA-3.0The Inter-american Development Bank (IDB) recently released the first set of results from its on-going, multi-year randomized evaluation of the impact of the OLPC project in Peru.
Experimental Assessment of the Program "One Laptop Per Child" in Peru (Spanish version here) is the first rigorous attempt to examine the impact of the largest '1-to-1 computing' initiative in a developing country.  This evaluation, done in concert with the Ministry of Education, looks at the ambitious program to provide computing resources to multi-grade rural elementary schools in some of the poorer communities of Peru.

Evaluating the evaluating of the Millennium Villages Project

Michael Trucano's picture

not all millennium projects are this neatly contained within clearly defined bordersWhen is the rigorous impact evaluation of development projects a luxury, and when is it a necessity?

This is a question asked in a new paper examing the Millennium Villages Project (MVP), a high profile initiative that, according to its web site, offers a "bold, innovative model for helping rural African communities lift themselves out of extreme poverty".

In the words of one of the authors of When Does Rigorous Impact Evaluation Make a Difference? The Case of the Millennium Villages, "We show how easy it can be to get the wrong idea about the project’s impacts when careful, scientific impact evaluation methods are not used. And we detail how the impact evaluation could be done better, at low cost."  The paper underscores the importance of comparing trends identified within a project activity with those in comparator sites if one is to determine the actual impact of a specific project.  This sentiment should come as no surprise to those familiar with an area of exploding interest in the international donor and development community -- that of the usefulness of randomized evaluations.

Evaluating the Millennium Villages: Reply to the MVP + Upcoming Seminar with Comments from Jeff Sachs

Gabriel Demombynes's picture

The following post was co-authored by Michael and Gabriel.

The Millennium Village Project (MVP) is an important, experimental package of interventions that the United Nations and Columbia University are testing in 14 villages across Africa. The MVP offers a tremendous opportunity to learn whether such interventions can catalyze self-sustaining growth and escape from extreme poverty. But the evaluation approaches currently being used cannot generate convincing evidence of the Project’s impacts. Without such evidence, it will be impossible to generate the billions of dollars needed to scale up the Project approach across Africa, as its proponents hope to do.

We have written a new research paper (summarized here and here) that proposes small and inexpensive modifications to the MVP evaluation approach that would make it possible to evaluate the Project’s impacts.

That paper has generated much discussion, including reports in the Financial Times and in a major newspaper in Kenya. The Project itself has issued a lengthy official response by Pronyk, McArthur, Singh, and Sachs. We welcome this public debate as a way to improve learning about what works in development. We answer below the main questions posed in the Project’s response, much of which rests on a basic misunderstanding.

Evaluating the Millennium Villages

Gabriel Demombynes's picture

Here’s the quick summary of a new working paper I have co-authored with Michael Clemens of the Center for Global Development:

When is the rigorous impact evaluation of development projects a luxury, and when a necessity? We study one high-profile case where it is a necessity: the Millennium Villages Project (MVP), an experimental intervention in rural Africa. We compare development trends inside versus outside the villages in three countries, and show that estimates of the project’s effects depend heavily on the evaluation method.

The impact evaluation currently planned by the MVP is unlikely to yield adequate estimates of its effects on Africans in general, for five reasons we explain. But it is not too late to carefully measure the project’s effects, by making small and inexpensive changes to the next wave of the project.

Michael’s own blog post gives more details about the paper. The paper uses publicly-available data from the MVP mid-term evaluation report and Demographic and Health Surveys  (DHS). Field visits played no role in the study.

But after the study I found myself wanting to learn more about a couple of the places behind the statistics. So after we completed the analysis, during September 26-28, I took a trip with several World Bank colleagues to the western edge of Kenya. We visited two village clusters in Nyanza Province: first the MVP site in Bar-Sauri, and then the town of Uranga, 50 km to the west, which is not an MVP site.

Here’s a picture of me pressing the flesh with the kids at Nyamninia Primary School in Bar-Sauri:

Technology and Transparency

Shanthi Kalathil's picture

Say you're a civil society activist who uses online and mobile technology as a tool for greater accountability. Wouldn't you want to be able to call up a map of the world and easily find examples from other countries that might also be relevant for your work?
 
Turns out, you can. Recently, at the Internet at Liberty 2010 conference co-sponsored by Google and Central European University, I heard a presentation from the Technology and Transparency Network, which is an initiative of Global Voices and Rising Voices.  Click on the link, and you'll see that the Technology and Transparency Network's home page is a map of the world, where you can zoom in on individual projects in countries like Mexico, Sudan, Uganda, Cambodia and Hungary. 

The Fad of Financial Literacy?

Bilal Zia's picture

Financial literacy has become an immensely popular component of financial reform across the world. As a response to the recent financial crisis, the United States government set up the President’s Advisory Council on Financial Literacy in January 2008, charged with promoting programs that improve financial education at all levels of the economy and helping increase access to financial services. In the developing world, the Indonesian government declared 2008 “the year of financial education,” with a stated goal of improving access to and use of financial services by increasing financial literacy. Similarly, in India, the Reserve Bank of India launched an initiative in 2007 to establish Financial Literacy and Credit Counseling Centers throughout the country which would offer free financial education and counseling to urban and rural populations. The World Bank also hasn’t been missing out on the trend – it recently approved a $15 million Trust Fund on Financial Literacy. 

But what do we know about financial literacy? Does it work, and if so, through what mechanisms? Despite the money being ploughed into financial literacy programs, we know very little to address these important questions. While it is true that there is a large and growing body of survey evidence from both developed and developing countries that demonstrate a strong association between financial literacy and household well-being, we are still in the process of learning whether this relationship is causal.

The $35 laptop?

Ryan Hahn's picture

The Indian Express is reporting that India's Ministry of Human Resource Development is set to launch a $35 laptop:

Looking as stylish as a large i-phone, this $35 “low-cost computing-cum-access device” is a 5/7/9 inch touchscreen gadget packed with internet browsers, PDF reader, video conferencing facilities, open office, sci-lab, media player, remote device management capability, multimedia input-output interface option, and multiple content viewer.

Seeking your questions for the roundtable on the role of the state in competitiveness

It is less than a week before the Promoting Competitiveness in Latin America and the Caribbean, hosted by the World Bank and the Mexican Ministry of Economy in Mexico City. More than 215 people have registered, including attendees from more than 20 different countries.

Doing industrial policy with limited government capacity

Ryan Hahn's picture

Responding to a post on Duncan Green's excellent From Poverty to Power blog, World Bank Chief Economist Justin Lin argues that his proposal for an updated version of industrial policy does not assume exceptional talent on the part of government institutions:


Pages