The Economist carried a couple of stories recently about how two hitherto major institutions in my home country (newspapers and pubs) have been forced to adapt in the face of changes in public preferences. Many didn’t—as a result newspaper circulation and pub numbers have both fallen dramatically. The newspapers and pubs that did survive operate very different business models from the newspapers and pubs in existence even 10 years ago.
Some data I’ve assembled make me wonder whether—like the newspaper and pub—the development-agency flagship might not also be an institution in need of reform.
The flagship
Most big development and international agencies have a flagship. The World Bank launched its World Development Report in 1978. The IMF’s World Economic Outlook started two years later. The UNDP launched its Human Development Report in 1990, and WHO followed with its World Health Report five years later. Several other UN agencies have annual or periodic flagship reports too.
The production of all the reports involves teams assembled for the purpose, comprising a mix of senior and junior staff and consultants. Most seem to take around a year to produce and disseminate, and are presumably therefore quite costly. Most focus on a specific theme, but also take the opportunity to present a roundup of new data though the data are usually available elsewhere too and are not always produced specifically for the report. All have a broad audience in mind, and aim at creating new knowledge in addition to synthesizing existing knowledge. All seek to influence the way policy makers, the public and scholars think about the chosen topic.
Given the costliness of these reports, it makes good sense to ask whether they are indeed influencing people, and whether if they have done so in the past they continue to do so today. This isn’t easy, but the huge volume of data on the internet gives us some clues about influence—and about interest which seems likely to be a precondition to influence.
Gauging interest
Fig 1 shows a 20-week moving average of the number of internet searches for four flagship reports that I assembled using Google Trends. The data are normalized with the first-entered search term averaging 1 over the seven-year period, and the other searches adjusted accordingly. On average the search frequency for the WDR was 60% that of the HDR, while the frequency of searches for the WHR and WEO were 31% and 21% of the HDR. Searches for the HDR have dropped considerably since 2004. Searches for the WDR have declined as well though less spectacularly. By contrast searches for the WEO increased during 2007 and stayed at the same level thereafter. Searches for the WHR started falling at the start of 2006 after an initial rise.
The internet search data suggest there’s been something of a waning interest in at least three of the flagships. This may reflect what seems to be a waning interest in international development. Fig 2 shows that searches for “international development” have fallen throughout the period, as have searches for related terms. As Fig 3 shows, German cars haven’t suffered the same fate: in fact, interest in the Mini—which has been made by BMW since 2001—has grown since the new model was launched in 2007. And searches for new technologies have risen pretty much continuously during the period (Fig 4). So if people had been growing ever more interested in international development and flagship reports, I think the data would have picked it up.
Gauging influence
Waning interest doesn’t necessarily imply waning influence. But how to measure influence? Martin Ravallion and I have argued for assessing influence on development thinking using citations in documents captured by Google Scholar. Unlike the Web of Science, Google Scholar captures not just established journals but also new journals, online journals, books, dissertations, and technical reports. Fig 5 shows for each year the number of hits in Google Scholar for each of the flagships. The year refers not to the year the flagship was published but the year of the article or book citing it. The charts don’t distinguish between flagships from different years. So it’s not surprising that in 1990 the WDR was getting more citations than the HDR, because by 1990—the first year of the HDR—12 WDRs had already been published. And it’s not surprising if citations keep going up. A fairer comparison across flagships and over time is to adjust citations in any year by the number of flagships to date to get an average figure per report. This chart (Fig 6) suggests that the influence per report continued to grow for all four flagships through to around 2003 but has since fallen off except in the case of the WEO.
Getting at influence more broadly—among policy makers and the public at large—is harder. Coverage in the media is one way. Fig 7 shows trends in Google News items for the four flagships. Around 2000 the WDR and WHR got left behind by the HDR and WEO. Then the “great recession” put the WEO on a trajectory reminiscent of Facebook’s trajectory in Fig 4. Of course news items reflect not just interest and influence but also the efforts of communications officers of the agencies. The UNDP might have just gotten much more media-savvy very quickly. And of course in any case Google News data don’t get behind the scenes and tell us whether people in power had their thinking fundamentally altered by a flagship report. We know the 1993 WDR shocked Bill Gates into setting up the Bill and Melinda Gates Foundation. But there’s no blip in the Google News data reflecting this.
The last five years—and the future
I like the Gates WDR story. But at heart I’m a numbers person. I find it striking that when you look at Figs 1 and 2 and the comparable period of Fig 6 you see a waning of interest in flagship reports and of their influence among academics and report-writers. When British newspaper- and pub-owners saw waning interest in their wares, they did a radical rethink. Is it time for development agencies to do a rethink on their flagships? Is the flagship model outdated? Could a different model help reverse the apparent decline in interest in international development?
Join the Conversation