Syndicate content

The Crucial Role of Connectors in Disseminating Research Results

David Evans's picture
A couple of weeks ago, I came across a fresh World Bank working paper (Doemeland & Trevino 2014) that examined downloads and citations for World Bank policy reports. The paper reports that 31 percent of policy reports have never been downloaded and 87 percent have never been cited. I thought this was an interesting statistic and – since it was publically available – I opened up Twitter and tweeted it. To the statistic, I added the question as to whether we needed “more relevant reports or better dissemination,” hoping to spark a conversation. At that moment, I had 40 followers on Twitter. In case you’re not familiar with Twitter, 40 followers is not very many. The median number of followers among people who have tweeted in the last 30 days is about 61.
 
Later that day, another Twitter user sent out a “modified tweet”. He quoted part of my tweet (with appropriate sourcing) and added his own (less positive) commentary. This user, Justin Sandefur, has about 2,600 followers. The next day, Tyler Cowen, who co-authors the popular economics blog Marginal Revolution, posted the statistics with a reference to Sandefur’s tweet. The post had 30 comments as of this morning, so we know people read it.

Christopher Ingraham read the post on Marginal Revolution and went on to post the statistics on the Washington Post’s Wonkblog with a more positive interpretation, highlighting that this is likely a problem for Washington think tanks broadly and commending the Bank for examining and publishing the numbers. He had 55 comments as of this morning; again, so we know people are reading it. Ingraham’s blog post was republished in the Sydney Morning Herald, excerpted on Slate, and referenced elsewhere. (A blog dedicated to “content access”, Xillio.com, even offered the Bank some tips on increasing access to our reports; Thanks, Xillio!)

Since all this has happened, I’ve had several conversations that highlight how these statistics may be misleading. Citations aren’t really the right measure of impact for policy reports, since government policy makers aren’t writing citation-heavy academic papers (thankfully!). Even downloads offer an underestimate: Many of these reports are emailed or hand-delivered directly to the policy makers most likely to use them (i.e., for a report on health in Malawi, to the Minister of Health in Malawi and her staff).

As Doemeland and Trevino write, “Many policy reports were … prepared to assess very specific technical questions or inform the design of lending operations.” The majority of these are not research working papers but rather reports like “Pakistan - Finding the path to job-enhancing growth” and “Education in Oman: The Drive for Quality.” Still, there is clearly room for improvement. But what I take away from this experience is less about the statistics themselves and more about how to get research results into public debate. (I admit, I’d have preferred to learn this with some of my own research results, say on how to reduce poverty in Tanzania, but maybe next time.)

This is a relatively uncommon situation, where we can clearly trace the growth in awareness from my minimally circulated tweet all the way to the Washington Post and beyond. We can’t rule out that the stat would have gone viral anyway; maybe Sandefur would have encountered it some other way. But we at least see clearly the distribution path that this piece of evidence followed.

From this, we see the importance of “connectors” in getting our knowledge work into the public dialogue. A connector, per Malcolm Gladwell’s book The Tipping Point, is a person who connects with lots of other people. Sandefur is a connector, with more than 2,600 followers, putting him above the 98th percentile of active Twitter users. Cowen is also a connector.

If you want to get your results known, get them into the hands of connectors. Oh, and it doesn’t hurt if the results are interesting. (As of early this week, Doemeland & Trevino had more than 850 downloads, which puts them on the far right tail of World Bank policy reports, per the figure from their paper.)
 

Bonus reading: The population of reports that Doemeland & Trevino draw on is available at the World Bank’s Documents & Reports archive, under “Economic & Sector Work.”

 

Comments

Submitted by Bobbi Gray on

David, I agree about your point above. Within the microfinance field, there has been a lot of discussion about bridging the divide between practitioners and researchers. I think there are a few issues to consider that do relate to connectors, but there are also other barriers. Language is one of them. Research is often published in the native language of the author and rarely translated. It's difficult to take a long research paper in English and try to share it with practitioners in the field when they only speak French or Spanish. The research also needs to be transformed from an academic paper into something manageable for those who can use it as well. For example, practitioners don't care about the formulas used to measure impact or change. Authors might consider how they might turn their research paper into a manageable research brief that could be translated. As a researcher within a technical assistance provider, I also find it frustrating that I might run across a fairly relevant study years after it's published. The World Bank blog is one way I try to stay current on what is being talked about within academic circles...but it's often not enough to stay on top of what is being published. Freedom from Hunger tries to play a connecting role between research and our practitioner partners, but as indicated above, this is a bit tedious when you can't provide any documentation in another language. We know we can also do better in playing a connecting role and have discussed different ways to do this, but just haven't found the right mechanism yet to make this a fluid process.

This language issue is really important. Reports written in Spanish and French by academics will often come out with their abstracts translated into English, but it's less common for an Executive Summary put out originally in English (unless the target country of influence is non-English speaking). Putting out the info in multiple languages (and figuring out how to disseminate THAT information effectively) would clearly increase application in the field.

Submitted by Paul Cadario on

Bob Sutton at Stanford talked about this over a decade ago. He was one of the first to look at network effects in organizations, and point out the value of 'long links' in sharing knowledge. This is after he co-authored a book "the knowing-doing gap", which was about the Bank but didn't identify the place. It would have been before your time, but you'd recognize it.

Submitted by Bobbi Gray on

Agreed..it would at least help get it one step closer to the people who can actually use the information and act on it. I also think the periodic synthesis of various related research topics also helps, because this also aids in helping practitioners who are not convinced of research coming from other countries, see that findings can be similar despite differences in context. The paper recently published by CGAP (http://www.cgap.org/publications/financial-inclusion-and-development-recent-impact-evidence) was a nice succinct summary of research on financial inclusion--but it is still only available in English.

Submitted by Tim France on

Many thanks for this insightful piece David. I am pleased to find I was an unwitting link in your chain.

One thought jumps out at me when you talk about the role of connectors (which I concur with by the way): What's different?

My perspective draws from the global health arena, but I am sure it's shared by many...

Until recently, mechanisms through which affected countries, communities and their people have been able to directly share the most relevant lessons and experiences with one another have been limited by the costs of communications and travel. Even the exchange of basic health-related information has relied heavily on international organizations as intermediaries. It still does.

But what your analysis reveals is that the disintermediating potential of global electronic communications is finally happening.

And it is not simply that institutional intermediaries are being replaced by individual 'connectors'. The essential requirement for connectors *and* comment also challenges the very idea that information has an inherent value in itself – the foundation of the hegemonic “we have the essential information, and you need it” models. This has to be a change for the better.

The World Bank report readership survey and your mapping of this communication chain are more than interesting: They are rare, inconspicuous tell-tales that show us before anything else that the wind is changing.

It would be interesting to map over time this shift from institutional hegemony towards a shared influence between individuals and institutions, Tim. Thanks for this.

Submitted by Joseph Ngwegwe on

Innovating Research findings dissemination for effective results

How we share information in our communities should largely determine a strategy to communicate policy information. If you quickly examine our daily life how many of our population accesses newspapers, Radio, television and nowadays social networks? How do we share sensitive information? What factors determines the speed at which information spreads? Fundamentally why do people share information anyway (especially from one person to another), what motivates you to share particular information and not the rest of what you’ve come across? When developing a communication strategy these are some of the questions we should be seeking answers to.

Policy information must be categorized and digressed to meet diverse targets and that will be determined by what do we exactly want this target group to do once they know about these facts. To influence policy practices our anchor target as you (David) have pointed out in your opening article are the policy designers and implementers. I don’t want to believe that these people are not aware of research insights we produce about policies. In a worst case scenario even if they haven’t seen a certain report at least they know where to get one if in need. The biggest challenge with our policy makers lays in their propensity and reaction to risks. Those of us who do research would want the findings to be of use, we would want policies are developed on the basis of these evidences. However, the policy development processes in the government are bureaucratic and in most cases politically influenced. My point here is that if we see nothing happening as to the effect of a certain powerful study we should not feel so much disappointed about it. Perhaps our strategy should be to use the Pareto principle, i.e. amongst the studies we have made what are the 20% that can lead into 80% of the envisaged results? Does every study we make carry with it same level of importance and sensitivity?

How communication flows in the communities depends on many facets, major one being culture (mainly language factor), level of education, individual purchasing power, age, gender etc. The most effective method to share information is still word of mouth through the physical social networks i.e. person to person, usually at home, in drinking places, in public transport, at work places during break hours, in formal meetings etc. The current technologically supported social networks are basically complementing. The reason why the social networks like Twitter, Facebook and the mobile applications e.g. SMS, Viber, Whatsapp, etc have become ubiquitous is because they almost fit into the traditional ways of communication. People who have never been trained in using computer applications are finding it easy to work with extremely sophisticated technologies and communication has been made simple.

For us to be successful we must fit our communication strategy into these innovations and we have to do it in a way that makes people feel excited not only to know about what we trying to convey but also thrill them to share with colleagues and friends. We must have a place to begin, you call them ‘connectors’ I call them ‘Anchors of social networks’. Who are they? These are individuals who have won hundreds of thousand or thousands of followers in their networks they range from artists to politicians, when they post news in their networks many followers actively engage in the discussions. These are everywhere in the country and around the global. We would need therefore to engage them and test whether they can make sense out of our research findings. Research findings dissemination needs to explore these networks and find convenient and exciting ways to reach their audiences more effectively.

Great post, Dave. Your comments and observations about the "crucial role of connectors" are, as they say, 'spot on'.

One comment, for what it might be worth. You note that "The paper reports that 31 percent of policy reports have never been downloaded and 87 percent have never been cited." Let's leave aside for a moment the issue of citations as a proxy for value and influence (and I'll assume that the 87% of reports that have never been cited include the 31% that have never been downloaded while I'm at it :-) ).

One way to look at the figure that 31% of reports have never been downloaded is to conclude (as many commenters seem to have done) that this suggests that they aren't of value. When I first saw this figure, however, I immediately asked myself: How is this even possible? If these reports were downloaded only a few times -- *that* I can certainly believe (for many of the reasons discussed in the paper). Even if they were only downloaded one time. But ... never? Never? This makes me wonder how find-able these reports are. Don't people vanity-Google themselves and make sure their papers appear and/or send links to their papers so that others can download them? Don't people run bots to download lots of these things so that they can be mirrored/hosted elsewhere? (For a few years there was a WB paper that I would reference but I would direct people to a link on another web site, as I could never find the paper on the Bank's own site, although I was assured that it was indeed there.)

While it is perhaps a small and rather trivial thing, and for what it's worth, I actively encourage other groups to mirror my blog posts, to re-post them in other fora, on listservs, etc. in full. One year I apparently had the most downloaded article on one of these third party sites. When I went and checked the official World Bank stats, very few people had downloaded the article from the Bank itself. It might just be that this third party site played the role of connector better than the Bank's site ...or better than the Bank's search engine ... or better than other search engines could when attempting to index the Bank's site. I used to regularly direct some people to a few World Bank reports that someone had for some reason re-posted on Scribd.

'Connectors', as you rightly suggest, can play important roles in helping people find information that might be of interest or relevance. So, obviously, can things like search engines and social media. I long ago stopped using the World Bank's search engine because it rarely helped me find what I was looking for. Instead, I would use things like Google to find things on the World Bank's own web site, and would ask colleagues for help in trying to locate documents on the World Bank's Intranet.

One implication that many folks seem to have drawn is that, because a report wasn't downloaded (or, seen from another perspective, hadn't yet been downloaded). This figure of 31% might well represent an indictment of the relevance and/or quality of the relevant papers. Might it might also reflect a failure of IT?

(Another possibility is that, in an era of increased openness, lots of more stuff if being made available. Lots of stuff that used to be hidden behind corporate firewalls at the World Bank are now being made available as part of the Bank's Access to Information policy. Maybe a lot of this stuff is of low quality, or questionable relevance. But if the cost of making it available digitally is near zero, why not liberate this stuff and make it publicly available for (potential) download? Maybe no one else is interested in it. Maybe it is isn't helpful or useful to anyone today. But if we can lengthen the long tail with very little effort, why not do so? We should be careful that our metrics are not informed by past notions and practices of scarcity that may no longer always apply like they used to.)

Add new comment