Big data and development: “The second half of the chess board”

|

This page in:

Do you think Fortune 500 CEOs care about Africa? In the past, frankly, with the exception of oil and gas giants, they didn’t. But this is changing… and fast.

This week, IBM is opening its Africa innovation hub in Nairobi. To demonstrate the significance of the occasion, IBM has brought along all its senior team, led by CEO Ginni Rometty (named #1 most powerful woman in business by Forbes in 2012). Like other ICT companies, IBM wants to ride the wave of Africa’s ICT revolution. In this area, Africa has not only been catching up with the West, but is in fact overtaking it in areas such as mobile money.

This important engagement comes at a time when many anticipate a global tipping point in computing and knowledge generation. A new wave of innovations, they argue, can come about for two reasons. First, the availability of data is rising exponentially. Data volume, velocity and variety are growing at an astounding rate. Today, 90 percent of the world’s data is only two years old. While most of it is unstructured and difficult to search, efforts underway – including through ‘Open Data’ initiatives – will facilitate reusing and repurposing of previously hidden data.

Second, we are entering a new era of computing. “Super-Computers” are being set-up everywhere (including one in Nairobi), which can process an unimaginable amount of data. More importantly, qualitative breakthroughs should soon allow us to use computers to model the behavior of the human brain. This is also called “cognitive computing”, allowing systems to learn, adapt, and ultimately hypothesize and suggest answers. IBM’s Watson is one of these new computes already becoming famous after beating the most successful human players of the game “Jeopardy”.

A well-known episode in the history of chess tells the fable of a gullible king who was drawn into ruin for underestimating the exponential impact of initially small numbers. The king, as the story goes, offered a reward to the creator of a game of chess, and asked him to name his price. The wise inventor asked for nothing but one grain of rice to be doubled for each square in the chess board. The ruler, arithmetically unaware, promptly accepted the offer, which he was unable to honor for the simple reason that it amounted to a billion tons of rice, double of the then world production. Put differently: If you were to take the same bet and start with one cent, the final amount would be 2,636 times larger than world GDP (see figure).

Figure - The exponential impact of small numbers (Click on the graph to see it larger)

Source: Toru Nishiuchi and Author’s illustration

How is this story relevant to development? Many development problems are more complex and important than chess. Think of predicting the weather, understanding food price patterns, or anticipating diseases. Even though our world is flush with data, its accuracy remains doubtful and its usefulness untapped. Basic facts, such as the exact size of a country’s economy or the number of poor people, remain uncertain in many countries, especially in Africa. A reading of this paradox is that we are still in the “first half of the board” waiting for the exponential impact of data to unfold.
 
But we are getting close to the tipping point and soon the marriage of knowledge and technology could put us in a position to decisively change the lives of people in poor countries. High-frequency and better organized data can help us calculate social and economic conditions faster. Africans should not have to wait for a decade after a survey is fielded to adjust their country’s GDP or estimate poverty numbers. For example, Kenya will need to wait for almost ten years to get new poverty estimates (the last survey was done in 2005 and the next one will only be launched after the elections).
 
In the future, these estimates should take weeks to produce. With high quality and frequency data, we can work out new development solutions, which simply don’t exist today.
Open Data is democratizing development solutions: Making data accessible in downloadable, machine-readable formats to larger groups of people can enable innovative solutions. Poor countries can leapfrog in the process, because they have much greater incentives to solve their development problems. A large number of “Techpreneuers” have already started to use data and ICT capacity in novel ways to create development solutions. One example is the Code4Kenya initiative, which embeds ICT- and data-literate Code4Kenya “fellows” in host media and civil society organizations to help them build open data applications that provide citizen-friendly information on health, education, and county development patterns.

Like the proverbial inventor in the chess story, they may bankrupt the old development institutions in the process, unless the latter adapt and ride the wave of change.

-------------------------------
Follow Wolfgang Fengler on Twitter@wolfgangfengler.

Millie
February 08, 2013

Wolfgang, great post!! :) I like your perspective on a blend of knowledge and technology, i think that way too often we tend to focus on technology for solutions that are top down and hardly fit local contexts, though this is changing. I wanted to shamelessly use this opportunity and the topic you wrote about to showcase the work we are doing, well in partnership with the World Bank, on big data and operational effectiveness of global development organizations- in a sense trying to do what you suggest (integrate knowledge and technology) for the purpose of investing aid funds more effectively and transparently.

Our idea is to organize a series of data dives (http://europeandcis.undp.org/blog/2013/01/11/can-big-data-help-deliver-…) where we would bring our organizational data sets to programmers and ask them whether we could use the data to answer some important questions (can we predict likelihood of corruption in projects, what leads to under-delivery of certain projects, are there any companies that tend to win majority of contracts in any given sector and what are their linkages to other suppliers, etc). We tried to take a stab at few of these issues two weekends ago, looking at the World Bank data set of major awarded contracts from 2007-2013 and came up with some really interesting results (http://europeandcis.undp.org/blog/2013/01/31/big-data-and-development-o…).

i think we're only at the beginning of understanding the incredible value hidden in torrents of operational data that global development projects are creating every day... data that can significantly improve targeting of the aid, risk management, and ultimately poverty reduction and sustainability.

Again thanks so much for a super interesting post, really enjoyed reading it!!
@ElaMi5

Rwakakamba Morrison
February 11, 2013

In Uganda, old cooperative socities have alot to catch up with. They are yet to harness the power of technology (10400000 mobile phones, 3.2 million net browsers and 228 FM stations) to govern markets of fresh and value added products. Indeed Kasenda farmers have abandoned old cooperatives and are using mobile phones to buy inputs and sell their bananas- see http://allafrica.com/stories/201210290433.html

Zohir Uddin
February 11, 2013

Greetings,

Its warming to hear that large tech giants such as IBM and others are beginning to take notice of the potential for innovation across some corners of Africa. Granted, huge swathes of the continent still remain largely tech-poor, yet iHubs such as Nairobi are leading the way.
I do hope that IBMs investments in Kenya result in some long-awaited technology transfer and knowledge sharing with local start ups and domestic entrepreneurs.
I'm eager to watch this space in a few years time.

Zohir Uddin.

David Matsinhe
February 14, 2013

The article raises good points. It also demonstrates that soon we will witness trends of zombie and irrelevant institutions that will be kept alive artificially. A lot of these international development institutions will be zombified by these developments. Unless they change, of course. This includes the World Bank among others.

Wolfgang
March 02, 2013

Thanks much Millie,

I very much look forward to joining the "data dive" at the World Bank from March 15-17:
http://datakindworldbank-es2002.eventbrite.com/?rank=1#

Hope to see you there, too

Wolfgang