Syndicate content

Add new comment

Big data and development: “The second half of the chess board”

Wolfgang Fengler's picture

Do you think Fortune 500 CEOs care about Africa? In the past, frankly, with the exception of oil and gas giants, they didn’t. But this is changing… and fast.

This week, IBM is opening its Africa innovation hub in Nairobi. To demonstrate the significance of the occasion, IBM has brought along all its senior team, led by CEO Ginni Rometty (named #1 most powerful woman in business by Forbes in 2012). Like other ICT companies, IBM wants to ride the wave of Africa’s ICT revolution. In this area, Africa has not only been catching up with the West, but is in fact overtaking it in areas such as mobile money.

This important engagement comes at a time when many anticipate a global tipping point in computing and knowledge generation. A new wave of innovations, they argue, can come about for two reasons. First, the availability of data is rising exponentially. Data volume, velocity and variety are growing at an astounding rate. Today, 90 percent of the world’s data is only two years old. While most of it is unstructured and difficult to search, efforts underway – including through ‘Open Data’ initiatives – will facilitate reusing and repurposing of previously hidden data.

Second, we are entering a new era of computing. “Super-Computers” are being set-up everywhere (including one in Nairobi), which can process an unimaginable amount of data. More importantly, qualitative breakthroughs should soon allow us to use computers to model the behavior of the human brain. This is also called “cognitive computing”, allowing systems to learn, adapt, and ultimately hypothesize and suggest answers. IBM’s Watson is one of these new computes already becoming famous after beating the most successful human players of the game “Jeopardy”.

A well-known episode in the history of chess tells the fable of a gullible king who was drawn into ruin for underestimating the exponential impact of initially small numbers. The king, as the story goes, offered a reward to the creator of a game of chess, and asked him to name his price. The wise inventor asked for nothing but one grain of rice to be doubled for each square in the chess board. The ruler, arithmetically unaware, promptly accepted the offer, which he was unable to honor for the simple reason that it amounted to a billion tons of rice, double of the then world production. Put differently: If you were to take the same bet and start with one cent, the final amount would be 2,636 times larger than world GDP (see figure).

Figure - The exponential impact of small numbers (Click on the graph to see it larger)

Source: Toru Nishiuchi and Author’s illustration

How is this story relevant to development? Many development problems are more complex and important than chess. Think of predicting the weather, understanding food price patterns, or anticipating diseases. Even though our world is flush with data, its accuracy remains doubtful and its usefulness untapped. Basic facts, such as the exact size of a country’s economy or the number of poor people, remain uncertain in many countries, especially in Africa. A reading of this paradox is that we are still in the “first half of the board” waiting for the exponential impact of data to unfold.
But we are getting close to the tipping point and soon the marriage of knowledge and technology could put us in a position to decisively change the lives of people in poor countries. High-frequency and better organized data can help us calculate social and economic conditions faster. Africans should not have to wait for a decade after a survey is fielded to adjust their country’s GDP or estimate poverty numbers. For example, Kenya will need to wait for almost ten years to get new poverty estimates (the last survey was done in 2005 and the next one will only be launched after the elections).
In the future, these estimates should take weeks to produce. With high quality and frequency data, we can work out new development solutions, which simply don’t exist today.
Open Data is democratizing development solutions: Making data accessible in downloadable, machine-readable formats to larger groups of people can enable innovative solutions. Poor countries can leapfrog in the process, because they have much greater incentives to solve their development problems. A large number of “Techpreneuers” have already started to use data and ICT capacity in novel ways to create development solutions. One example is the Code4Kenya initiative, which embeds ICT- and data-literate Code4Kenya “fellows” in host media and civil society organizations to help them build open data applications that provide citizen-friendly information on health, education, and county development patterns.

Like the proverbial inventor in the chess story, they may bankrupt the old development institutions in the process, unless the latter adapt and ride the wave of change.

Follow Wolfgang Fengler on Twitter@wolfgangfengler.