Syndicate content


Promoting literacy with mobile phones in rural Papua New Guinea

Michael Trucano's picture
hey, my ears are ringing -- might that be the Ministry of Education calling with today's lesson?
hey, my ears are ringing --
might that be the Ministry of Education
calling with today's lesson?

Last year I spent some time in Papua New Guinea (or PNG, as it is often called), where the World Bank is supporting a number of development projects, and has activities in both the ICT and education sectors. For reasons historical (PNG became an independent nation only in 1975, breaking off from Australia), economic (Australia's is by far PNG's largest export market) and geographical (the PNG capital, Port Moresby, lies about 500 miles from Cairns, across the Coral Sea), Australia provides a large amount of support to the education sector in Papua New Guinea, and I was particularly interested in learning lessons from the experiences of AusAid, the (now former) Australian donor agency.

For those who haven't been there: PNG is a truly fascinating place. It is technically a middle income country because of its great mineral wealth but, according to the Australian government, "Despite positive economic growth rates in recent years, PNG’s social indicators are among the worst in the Asia Pacific. Approximately 85 per cent of PNG’s mainly rural population is poor and an estimated 18 per cent of people are extremely poor. Many lack access to basic services or transport. Poverty, unemployment and poor governance contribute to serious law and order problems."

Among other things, PNG faces vexing (and in some instances, rather unique) circumstances related to remoteness (overland travel is often difficult and communities can be very isolated from each other as a result; air travel is often the only way to get form one place to another: with a landmass approximately that of California, PNG has 562 airports -- more, for example, than China, India or the Philippines!) and language (PNG is considered the most linguistically diverse country in the world, with over 800 (!) languages spoken). The PNG education system faces a wide range of challenges as a result. PNG ranks only 156th on the Human Development Index and has a literacy rate of less than 60%.  As an overview from the Australian government notes,

"These include poor access to schools, low student retention rates and issues in the quality of education. It is often hard for children to go to school, particularly in the rural areas, because of distance from villages to schools, lack of transport, and cost of school fees. There are not enough schools or classrooms to take in all school-aged children, and often the standard of school buildings is very poor. For those children who do go to school, retention rates are low. Teacher quality and lack of required teaching and educational materials are ongoing issues."

[For those who are interested, here is some general background on PNG from the World Bank, and from the part of the Australian Department of Foreign Affairs and Trade that used to be known as AusAid, a short report about World Bank activities to support education in PNG from last year and an overview of the World Bank education project called READ PNG.]

If you believe that innovation often comes about in response to tackling great challenges, sometimes in response to scarcities of various sorts, Papua New Guinea is perhaps one place to put that belief to the test.

Given the many great challenges facing PNG's education sector,
its low current capacity to meet these challenges,
and the fact that 'business as usual' is not working,
while at the same time mobile phone use has been growing rapidly across society,
might ICTs, and specifically mobile phones,
offer new opportunities to help meet many long-standing, 'conventional' needs
in perhaps 'unconventional' ways?

A small research project called SMS Story has been exploring answers to this question.

#4 from 2013: Numbers Are Never Enough (especially when dealing with Big Data)

Susan Moeller's picture

Our Top Ten Blog Posts by readership in 2013
This post was originally published on January 8, 2013

The newest trend in Big Data is the personal touch.  When both the New York Times and Fast Company have headlines that trumpet: “Sure, Big Data Is Great. But So Is Intuition.” (The Times) and “Without Human Insight, Big Data Is Just A Bunch Of Numbers.” (Fast Company) you know that a major trend is afoot.

So what’s up?

The claims for what Big Data can do have been extraordinary, witness Andrew McAfee and Erik Brynjolfsson’s seminal article in October in the Harvard Business Review: “Big Data: The Management Revolution,” which began with the showstopper:  “‘You can’t manage what you don’t measure.’”  It’s hard not to feel that Big Data will provide the solutions to everything after that statement.  As the HBR article noted:  “…the recent explosion of digital data is so important. Simply put, because of big data, managers can measure, and hence know, radically more about their businesses, and directly translate that knowledge into improved decision making and performance.”

Ups and Downs in the Struggle for Accountability – Four New Real Time Studies

Duncan Green's picture

OK, we’ve had real time evaluations, we’ve done transparency and accountability initiatives, so why not combine the two? The thoroughly brilliant International Budget Partnership is doing just that, teaming up with academic researchers to follow in real time the ups and downs of four TAIs in Mexico, Brazil, South Africa and Tanzania. Read the case study summaries (only four pages each, with full versions if you want to go deeper), if you can, but below I’ll copy most of the overview blog by IBP research manager Albert van Zyl.

By following the work rather than tidying it all up with a neat but deceitful retrospective evaluation, they record the true messiness of building social contracts between citizens and states: the ups and downs, the almost-giving-up-and-then-winning, the crucial roles of individuals, the importance of scandals and serendipity.

What is a Theory of Change and How Do We Use It?

Duncan Green's picture

I’m planning to write a paper on this, but thought I’d kick off with a blog and pick your brains for references, suggestions etc. Everyone these days (funders, bosses etc) seems to be demanding a Theory of Change (ToC), although when challenged, many have only the haziest notion of what they mean by it. It’s a great opportunity, but also a risk, if ToCs become so debased that they are no more than logframes on steroids. So in internal conversations, blogs etc I’m gradually fleshing out a description of a ToC. When I ran this past some practical evaluation Oxfamers, they helpfully added a reality check – how to have a ToC conversation with an already existing programme, rather than a blank sheet of paper?

But first the blank sheet of paper. If you’re a regular visitor to this blog, you’ll probably recognize some of this, because it builds on the kinds of questions I ask when trying to understand past change episodes, but throws them forward. Once you’ve decided roughly what you want to work on (and that involves a whole separate piece of analysis), I reckon it’s handy to break down a ToC into four phases, captured in the diagram.

Videogames and Learning

Michael Trucano's picture
playing to learn?
playing to learn?

Not a week goes by where I don't receive an unsolicited email from a company touting the benefits of its new 'educational videogame'. Indeed, just last week I opened my inbox to find two separate emails proclaiming how two different mobile gaming apps were destined to "transform learning!!!" Now, in a lot of the cases, I must confess that I am not always sure why something is an 'educational game', and not just a 'game' (although if I am in a difficult mood, I might offer that in too many instances an 'educational game' is 'a game that really isn't much fun'). That said, there is no denying that videogames are big business around the world. So -- increasingly -- is education. Even most people who fear that potential negative effects of some (or even most) videogames on young people would, at the same time, acknowledge the promise and potential for videogames to offer enriching learning experiences. The history of the introduction of educational technologies is in many ways long on promise and potential, however, and short on actual evidence of how they impact learning in tangible and fundamental ways.

Much is made of the potential for ICTs to be used to promote more personalized learning experiences through the introduction of various types of ICT-enabled assessment systems. For me, it has long seemed like the most powerful real-time learning assessment engines have been found in videogames, where actions (or inactions) are often met with near instantaneous responses, to which the player is then challenged to respond in turn. This feedback loop -- taking an action, being presented with information as a result, having to synthesize and analyze this information and doing something as a result -- might meet some people's definition of 'learning'. A good videogame engages its users so strongly that they are willing to fail, and fail, and fail again, until they learn enough from this failure that they can proceed with the game. Even where educational software is not explicitly labeled as a 'game', designers are increasingly introducing game-like elements (badges, achievement bonuses, scoring systems) as a way to promote user (learner? player?) engagement as part of a process known as 'gamification'.

The use of videogames for educational purposes, or at least in educational contexts, is far from an OECD or U.S. phenomenon. Whether I am visiting a school computer lab after hours in central Russia, an Internet cafe filled with students in Indonesia or standing behind some schoolgirls carrying phones between classes in Tanzania, 'educational' videogames seem to be nearly everywhere. Past posts on the EduTech blog have profiled things like the use of video games on mobile phones to promote literacy in rural India and EVOKE, an online game for students across Africa which the World Bank helped sponsor a few years ago. When I speak with young software entrepreneurs in Nairobi or Accra or Manila, they often talk excitedly about the latest educational game they are developing (for markets local and distant).

Do educational games 'work'?
Are they 'effective'?
And if so: How can they be used in schools?

Questions such as these are of increasing interest to scholars. Given both their potential for learning, and how aggressively videogames are being marketed to many education systems, they should be of increasing interest to educational policymakers as well. Some recent research brings us a little closer to a time when we can answer some of them.

Will Midterm Evaluations Become the Dinosaurs of Development?

Milica Begovic's picture

I argued a few months back that information we get from story-telling is fundamentally different to what we get from polls and surveys. If we can’t predict what’s coming next, then we have to continuously work to understand what has and is happening today. (See: Patterns of voices from the Balkans – working with UNDP)

Methods we’re all used to using (surveys, mid-term evaluations) are ill prepared to do that for us and increasingly act as our blindfolds.

Why stories?

As I started working through the stories we collected, this question has become even stronger.

To give you some background, we started testing whether stories could help us:

So What do I take Away from The Great Evidence Debate? Final Thoughts (for now)

Duncan Green's picture

The trouble with hosting a massive argument, as this blog recently did on the results agenda (the most-read debate ever on this blog) is that I then have to make sense of it all, if only for my own peace of mind. So I’ve spent a happy few hours digesting 10 pages of original posts and 20 pages of top quality comments (I couldn’t face adding the twitter traffic).

(For those of you that missed the wonk-war, we had an initial critique of the results agenda from Chris Roche and Rosalind Eyben, a take-no-prisoners response from Chris Whitty and Stefan Dercon, then a final salvo from Roche and Eyben + lots of comments and an online poll. Epic.)

On the debate itself, I had a strong sense that it was unhelpfully entrenched throughout – the two sides were largely talking past each other,  accusing each other of ‘straw manism’ (with some justification) and lobbing in the odd cheap shot (my favourite, from Chris and Stefan ‘Please complete the sentence ‘More biased research is better because…’ – debaters take note). Commenter Marcus Jenal summed it up perfectly:

Lant Pritchett v the Randomistas on the Nature of Evidence - Is a Wonkwar Brewing?

Duncan Green's picture

Recently I had a lot of conversations about evidence. First, one of the periodic retreats of Oxfam senior managers reviewed our work on livelihoods, humanitarian partnership and gender rights. The talk combined some quantitative work (for example the findings of our new ‘effectiveness reviews’), case studies, and the accumulated wisdom of our big cheeses. But the tacit hierarchy of these different kinds of knowledge worried me – anything with a number attached had a privileged position, however partial the number or questionable the process for arriving at it. In contrast, decades of experience were not even credited as ‘evidence’, but often written off as ‘opinion’. It felt like we were in danger of discounting our richest source of insight – gut feeling.

In this state of discomfort, I went off for lunch with Lant Pritchett (right – he seems to have forgiven me for my screw-up of a couple of years ago). He’s a brilliant and original thinker and speaker on any number of development issues, but I was most struck by the vehemence of his critique of the RCT randomistas and the quest for experimental certainty. Don’t get me (or him) wrong, he thinks the results agenda is crucial in ‘moving from an input orientation to a performance orientation’ and set out his views as long ago as 2002 in a paper called ‘It pays to be ignorant’, but he sees the current emphasis on RCTs as an example of the failings of ‘thin accountability’ compared to the thick version.

How to Evaluate Bias and the Messages in Photos

Susan Moeller's picture

Can you tell if a news outlet, an NGO or a government is picturing a person, an event or an issue fairly?  It can be very hard to assess visual “balance” when photos are scattered across a website, and appear sporadically over a span of time.  There may be an anecdotal impression that there is bias, but visual bias has been very difficult to document.

The social media site Pinterest is now making documentation possible.

Have you heard of Pinterest?  According to the site itself, it’s “a Virtual Pinboard” that lets users “organize and share all the beautiful things you find on the web. People use pinboards to plan their weddings, decorate their homes, and organize their favorite recipes.”  Doesn’t exactly sound like the kind of site that would help journalists or academics, governments or NGOs, does it?  But Pinterest is turning out to be a stealth tool for researchers.

Getting Evaluation Right: A Five Point Plan

Duncan Green's picture

Final (for now) evaluationtastic installment on Oxfam’s attempts to do public warts-and-all evaluations of randomly selected projects. This commentary comes from Dr Jyotsna Puri, Deputy Executive Director and Head of Evaluation of the International Initiative for Impact Evaluation (3ie)

Oxfam’s emphasis on quality evaluations is a step in the right direction. Implementing agencies rarely make an impassioned plea for evidence and rigor in their evidence collection, and worse, they hardly ever publish negative evaluations.  The internal wrangling and pressure to not publish these must have been so high:

  • ‘What will our donors say? How will we justify poor results to our funders and contributors?’
  • ‘It’s suicidal. Our competitors will flaunt these results and donors will flee.’
  • ‘Why must we put these online and why ‘traffic light’ them? Why not just publish the reports, let people wade through them and take away their own messages?’
  • ‘Our field managers will get upset, angry and discouraged when they read these.’
  • ‘These field managers on the ground are our colleagues. We can’t criticize them publicly… where’s the team spirit?’
  • ‘There are so many nuances on the ground. Detractors will mis-use these scores and ignore these ground realities.’

The zeitgeist may indeed be transparency, but few organizations are actually doing it.