A recent Policy Research Working Paper “Which World Bank Reports are Widely Read?” garnered a flurry of online coverage over the past week and a half.
Unfortunately, several reports misunderstood the paper’s conclusions. As fellow blogger David Evans pointed out yesterday, a few Tweets and stories implied that most Bank reports are not being used at all. That’s clearly not true. In fact, people across the world downloaded World Bank reports millions of times over the past two years. It is true, however, that certain technical, country-specific, and sector-specific reports (the only ones studied by the working paper) are much less widely read, or were not downloaded via the Documents and Reports database that the authors analyzed. Even if not downloaded, these reports were certainly delivered to the clients who commissioned them, and were often emailed to others, or disseminated the old-fashioned way by printing and hand distribution -- a common practice in many parts of the developing world where we work and where internet access is limited.
Given the scope of the World Bank’s knowledge assets, this is a complicated story. The Bank produces a wide variety of knowledge products, including annual global flagship reports (the World Development Report, the Global Monitoring Report, Global Economic Prospects, and Doing Business); independent evaluations; books on a wide range of development topics; periodic macroeconomic economic updates at the country, sub-regional and regional levels; comprehensive development databases such as the World Development Indicators; tools for poverty measurement and applied economic analysis such as PovcalNet and ADePT; journals such as the World Bank Economic Review; policy research working papers; annual reports; knowledge notes, and; specialized economic sector work (ESW) and technical assistance (TA) reports.
Altogether, our knowledge content received more than 3.4 million downloads during the past two years alone. Yet, the paper’s authors, Doerte Doemeland and James Trevino found that, while analysis had been done on the impact of Bank Group flagship reports and the influence of Bank-authored research papers, much less was known about the influence and reach of ESW and TA – the last group of papers on the list above. ESW and TA are often designed to meet a specific country client’s demands as part of operational work. Doemeland and Trevino focus specifically on the determinants of downloads and citations for ESW and TA between July 2008 and June 2012.
ESW and TA reports are the nitty-gritty of the World Bank's engagement with client countries, and most reach their (sometimes narrow) objectives. Doemeland and Trevino cite a survey which finds that the majority of government counterparts think that ESW and TA reports are effective at achieving their objectives, and more than three quarters of them have actually led to changes in policies, regulations, or institutions as mentioned in the text.
We know these reports are read by the clients who commissioned them and have an impact. But are ESW and TA reports having an impact beyond specific experts in country governments (say, with the education ministry in the case of a report on how to retain girls in middle school in a particular country)? Are they also of interest to experts in other countries, and can we package and share the results to ensure they are used beyond their initial narrow purpose? This is precisely the sort of question being discussed during the World Bank Group’s ongoing change process. For instance, our Global Practices and Cross-cutting Solution Areas will be operational July 1. Their purpose is to share development solutions across boundaries and eliminate sector and regional silos that too often leave the valuable knowledge contained in ESW reports unread by all but a few experts at the country or regional level.
While knowledge sharing takes place in many different settings—through seminars, presentations, blogs, and so on—the authors looked only at downloads and citations to gauge the use of these reports. Actually, they found that some “core” ESW and TA reports that covered multiple sectors in middle-income countries with large populations were extensively downloaded. Meanwhile, others that focused on specific technical questions received much less attention. In fact, the authors found that 31 percent of ESW and TA reports in their sample from the Documents and Reports Database (the only comprehensive database that was available when they prepared the report) were never downloaded.
To some extent, this was to be expected. Several of the ESW and TA reports Doemeland and Trevino studied were made public under the Bank’s Access to Information Policy and were never originally intended for wide dissemination. Others were designed to inform the Bank’s lending policies or summarize advice on very specific issues.
But their paper also found that reports with the stated objective of informing the public debate received significantly more downloads.
Our hope is that this kind of analysis can push us to find innovative ways to increase our impact and spread the Bank’s knowledge. Doemeland and Trevino discuss how more dissemination efforts may drive greater awareness, and ultimately greater downloads. For example, one author of a well-read ESW report told them how he and his team printed thousands of copies of their country policy report and placed them in libraries across the country. That’s laudable, but more actions are needed if we are to raise our game.
About four years ago the Bank launched an Open Agenda, whereby the institution now shares, free of charge, virtually all our research technical reports. As part of that, an Open Knowledge Repository, was established. This repository has significantly boosted discoverability and dissemination of all Bank reports and has had more that 3 million document downloads over the past two years. Currently about 3,500 of our ESW and TA reports are on the Open Knowledge Repository; just over 10,000 are on the Documents and Reports database.
It’s true that, in many cases, we need to become far better at getting this work into the hands of a wider set of practitioners, citizens, entrepreneurs, civil society experts and other development actors. We know our reports are already having an impact, but we should accelerate sharing of detailed knowledge across countries and sectors so that Bank expertise can be tailored and replicated in other countries, where needed.
We want knowledge to be absorbed and re-used across the World Bank Group, as well as by our development partners across the world. Such a solutions-oriented approach means more innovative outreach and dissemination. This is a big priority across the World Bank Group and is one of the issues we're taking on as part of our broader change process.
Join the Conversation