Who's listening to the "knowledge bank"?
This page in:
We now know quite a lot about the supply of research on development, and about the part the World Bank plays. We know that the World Bank publishes a lot, that most research in the world is by researchers in high-income countries, and that were it not for the Bank there would be far fewer journal articles about developing countries.
We know much less about the demand for development research and Bank publications in particular. We know that Bank publications get cited a lot in scholarly journals, books, and technical reports. What we don’t know is: who is reading and citing the Bank’s work?
An optimist would argue that Bank authors get read largely by people in developing countries, and this in turn helps move policy forward. A cynic would argue that the audience for Bank publications is largely made up of others in the North working on international development. Bank reports that end up in developing countries are, according to this view, at risk of being turned into papier-mâche masks for puppet shows. We call this the "puppeteers view".
So, who’s right?
Some Facts
Sales data suggest the bulk of Bank books get sold in high-income countries. But our friends in the Bank’s publisher’s office warn us against reading too much into these numbers. They point out that sales are often registered to distributors who operate in multiple markets.
And Bank books and working papers are increasingly being made available electronically. The Bank’s e-Library is available on a subscription basis to libraries, and many publications can be viewed for free at Google Books or in “widget” form at ISSUU and Scribd. Many can be downloaded in PDF format for free from the Bank’s Documents and Reports website.
The latter channel may be the Bank’s best bet for reaching a developing-country audience, and this is the channel that seems to have the best geographic usage data at the moment. With these data, we can do a simple test to see whether the optimist or puppeteer is right: compare population and download numbers for each of four country groupings: low-income, lower-middle income, upper-middle income, and high income. The optimist would be hoping to see the high-income country grouping’s population share fall short of its download share.
We assembled the numbers for World Development Reports (aggregating downloads of the 2004 report on service delivery, and the 2006 report on equity and development) and Policy Research Working Papers produced by the human development team in the Bank’s Development Research Group (DECHD) (aggregating across 25 randomly-selected papers from the last 2½ years). We also looked at the location of visitors to the DECHD website to see whether our developing-country audience might be “just visiting”.
The results in the chart below will disappoint the optimist and delight the puppeteer. High-income countries account for just 17% of the world’s population but nearly 75% of WDR downloads. They also account for around 70% of working paper downloads and website visitors. The DECHD team’s working papers and website do a slightly better job than the WDR of reaching lower middle-income counties, but the difference is pretty small.
.gif)
The Wrong Comparison?
The optimist might complain about our use of population shares as the comparator. The developing world has lower literacy rates, and even lower fractions of people with sufficient education to enable them to read a WDR. It also has fewer internet connections per 1000 population.
So, we also show on the chart the global distributions of university students, internet users, and people with high or advanced TIMMS math and science scores. (We applied TIMMS high/advanced rates among 15-year olds to the population as a whole, and assumed countries without TIMMS data had the same rates as others in the same income group.) On all these indicators, high-income countries account for less than 50% of the global total. These countries’ share of WDR downloads is almost 30 percentage points higher than their share of the world’s internet users.
So What?
So, it looks like the puppeteer is largely right. The high-income countries’ shares of WDR and working paper downloads are surely far higher than one would like.
This puppeteer’s viewpoint raises some questions, some rather awkward:
- Do other Northern institutions do better than the World Bank in reaching a developing-country audience? Or do all Northern authors end up writing for a largely Northern audience?
- If training and internet access aren’t binding constraints, why is the demand for Bank publications so heavily concentrated in high-income countries? Do developing country audiences think the Bank’s publications are irrelevant? Are there things that could be done to increase readership in the South?
- Does it make sense for the Bank and other Northern institutions to carry on producing knowledge about the South if the demand for the knowledge is located largely in the North? Or are there benefits to developing counties of the North’s publications (e.g. in policy making) that are being transmitted through channels that don’t entail in-country downloads?
- Do Bank researchers stand a better chance of connecting with developing-country audiences through a "wholesaling" approach, i.e. making tools available so as to facilitate "new learning in specific contexts"? Our chart shows that to date users of two of the Bank’s major analytic tools—ADePT and PovCalNet—as well as of LSMS household surveys also come largely from high-income countries. But challenges also lie ahead in increasing the use of Bank data in the developing world. Users of the World Development Indicators and viewers of the new Open Development webpage also speak with distinctly Northern accents.
- What then do the Bank and others need to do differently to "democratize development economics"?
You seem to be discounting the fact that a large number of Southerners attend Northern universities where they make up part of that HIC demand.
Thank you for tackling this subject - we owe it to our clients and ourselves to constantly question our effectiveness. Shining the light on the Bank's inability to reach priority audiences with its knowledge products is the first step towards improvement. Next is to increase accessibility, relevancy, and 'digestibility'.
Not really, Stuart. Developing-country students in the North presumably complete their studies at some stage, and return home. At least some of them are likely to become academics and government officials—two big target audiences for Bank publications. In any year, I suspect Southerners studying in the North are likely to be far outnumbered by previous cohorts of Southern students who returned home.
You asked very important question: who is audience of the WB research? Is the policy-maker in the developing countries or compatriot researchers in the North. If the audience is policy-makers in the developing world, how accessible is the research we produce --is another entirely different question that should also be asked. Another question is sometimes it is not obvious to me what value the paper adds to solving problems in development. Even our colleagues in the operations do think our research papers are too academic. One simple way to find out how research is used in operations and in low income countries, one could conduct survey just among the WB staff in operations and country offices and see if they use research in their projects. That survey would reveal the effectiveness and importance of research in developing countries.
My reaction is same as the others here -- can't assume that the university and grad students in high income countries aren't from developing countries or headed there. I also assume that those with high speed lines, printers and personal computers are more apt to download big documents than those who don't. Thanks for the research -- we need more of this digging down beneath the surface.
Dear Adam:
I come from a developing country and have been doing research about my country in Europe during the last years. I am a frequent user of the data base from the World Bank. However, I believe if you would check my statistics, I would never appear as an user from a developing county. Like me, as a Stuart pointed out, there are many researchers who could be misplaced in your data base.
You have to take in consideration that if I do a plain, simple google search about "poor countries". The first hits I will get would be from wikipedia. Would any of the documents from the World Bank show in that search? I'm highly convinced the answer is no. Even if I enter to the database from the World Bank, it is not easy to find the documents. Most of those documents are in english so, if I want to use them you have to assume: a) I have internet b) I know english c) My level of knowledge is high enough to understand it's content. How many persons fit that profile in developing countries?
As for the use of data, well, I guess up to the user how to apply it. You give a tool, it is difficult to assess if it will be used as a weapon.
For sure the WB is an important producer, or "dealer" of information and knowledge. As a Brazilian health sector worker, discussing and living, all the time, the effects of industries and technologic centers pressure (or crushing) the expectancy of life with quality by our people.
I´m not an optimist but you can help us more and more, using the same technologic strategy, only stay visible and properly inside the Southern living and suffering.
I have always thought a survey along the lines you decsribe would be a good idea. But bear in mind the chart refers not just to research working papers, but to the WDR and to Bank datasets. The WDR is intended to be very accessible, and to be a bridge between research abd policy. I was surprised to see that the Northern share of readers is higher for the WDR than for research working papers.
Fair point but... (1) As I said above, current Southern students should be far outnumbered by returnee students. Unless, of course, a very large country has only recently started sending students abroad in big numbers, and we haven't yet reaped the dividend of a larger home audience. (2) I was surprised to see the internet access data in comparison to the download data; they seem to suggest that connectivity may not be the constraint. (3) If connectivity is the constraint, it raises the question: How can we better reach our audience?
Thanks, Coello. It’s true as you say that researchers like you will be incorrectly counted as Northern, though I suspect that for everyone like you there are lots of compatriots working in-country. I totally agree with you on difficulty of navigating around the Bank’s website. The good news is that the website is going through a complete overhaul. Parts of the site have already been overhauled. The new data page, for example, is great: http://data.worldbank.org/. Lack of English is clearly a barrier. However, publications like the WDR are translated into many languages. Our WDR data include non-English versions.
Thanks Adam for raising this point. I also appreciate the angles that previous contributors have taking on the issue. However, I would like to bring up two sides that I think is what considering:
1. The Bank may want to consider alternative media of conveying or disseminating development research and knowledge in developing countries. A good example could be the use of film! I am aware that EAP region is currently doing something along that line in Mongolia. The Nigerian Movie industry - Nollywood, a home grown industry that sprang out of nothing and reputed to be the third-largest film industry worldwide with movies watched across the continent and beyond could be a good channel to deliver key development messages.
2. My second point is the issue of interest. For a group of people largely concerned about the basics of life, how much interest could there be in researching and consumption of researched materials. However, this could be improved by use of media that could be easily assimilated amongs the group as referred in my point above.
Talking among ourselves is a problem that we ourselves in the Bank perpetuate in part because, despite our nobler intentions, a primary audience and evaluator of our research and ESW is the internal one. That's why I think the work that the education sector is developing called SABER -- System Analysis and Benchmarking for Education Results is a good example of analytic work designed to serve clients' operational needs, across all types of countries. As a key tool to help implement the new Education Sector Strategy 2020, SABER is meant to communicate what matters (based on available evidence and practice) in each of the critical policy areas in education systems (teacher systems, assessment systems, etc.), describe this in ways that are clear, readily understood, and usable by counterparts, and develop diagnostic tools to help countries (or subnational jurisdictions) improve the efficiency and performance of their systems. The education sector is developing a website for SABER, and already has made headway in teacher systems work, as well as in other areas including, inter alia, Assessment Systems, and Workforce Development Systems. Links to the new strategy, and to SABER and the teacher work can be found at www.worldbank.org/education
You should also check who reads this blog. WB reports and papers are very technical and usually hidden deep inside the WB website. Blogs in general are meant to be easier to read and more readily found, and as is mentioned in the welcome header, this blog is also meant to increase the space for development dialogue.
I am from a developing country, and I found this blog through shared link on Facebook. However, it was from my circle of friends in the development field, which is a very small group of people.
I come from a developing country, Zambia. I am curious to know just how much effort the WB puts in incorporating works of scholars in developing countries. For the few of us that make an effort to read some of these documents, they not only tend to be too technical but too generalised, thereby diluting their relevance. If the bank's work is indeed for the greater good of the developing nations, interest in reading those publications could be enhanced if the Southerners themselves were given as much attention because they after all understand issues better than the Northerners whose experience on developing nations may only be based on arguments advanced by previous scholars without necessarily having been to any of the areas they write about. On the other hand, unfortunately, I do not think that politicians in places like my country even take time to read such documents even though I believe they should be the primary recipients because of their role in policy formulation. They change ministries as the president wishes and as such for those that do read, it is difficult to keep the momentum in implementing certain measures.
I think there is plenty of demand for WB publications. But 90% of the information people want to know from WB is still not available in a format that's easy for them to quickly understand. If WB wants to be a knowledge bank than it needs to build a system for its target audience to get the information it wants much more easily and efficiently than they have ever been able to.
Some of the papers has some great research but they are so hard to crack or just plain boring.
You need computer engineers to figure out a new platform to share information and you need good writers (snag a few from The Economist) to communicate all the research being generated by your top notch economist. And than you need professional web designers and MBAs with a focus on marketing to put this information out.
Interesting information and a very valid question to look into. Others have mentioned issues regarding the sample and language issues (what proportion of all these reports are available in other languages?). I would be interested in knowing if this survey would have had different results with data from the 1990s. Whereas at one point the Bank was one of the few organizations doing serious development research, it now faces competition from academia, the private sector and from other on line information sources. These sites and are more easily accessed using google or other browsers.
Many "development" workers are from Northern Countries. It is still inherently a top down notion. If the research came under a business model for example, there might be more interest from a broader spectrum of people.
In some countries data usage is still not benign or transparent. People have no reason to believe numbers or to believe that the information generated by them will bring positive benefits. More work needs to be done to broaden the understanding of and constructive use of data. Action Aid has done some of this, but much more could be done, for example in education, through working with stakeholders from school to central government levels. This links directly with the ability to measure meaningful results and to use data for policy change.
How often is WB research undertaken jointly with Southern institutions or taken to client country audiences for discussion rather than staying in house? I know it happens sometimes but it should underpin research efforts. The notion that it is the Bank that has the knowledge is perhaps an impediment to its use!
Eluned Schweitzer
I am a regular user of WB reseacrh papers and policy documents, not so much your paid publications. Reasons being - I am an indirect applicator of this information and I am not specialising in social science but work on IT and IT solutions for problems/perceived problems in the government space. Secondly, the developing or low income countries are indeed prioce sensitive. I speak from the Indian Experience and when we were in College we had what we called the EEE - Eastern Economy Edition of all publications that would probably cost an arm and a leg in the west even for westerners. Considering that you have the capability to deliver over the web, local printing could be a solution, leveraging developing exconomy costs., Secondly, if you didd see the EEE that i was mentioning above the qulaity of the covers and the paper and the printing are not too good, but it did and does continue to give me the content that I am looing for. If you can differentiate your offerings with the content being the same I am sure you will see a more usage in the developing or MIC/LIC as you call it.