The Impact of Blogs Part II: Blogging enhances the blogger’s reputation. But, does it influence policy?
This page in:
On Monday, we examined the impact of blogs on downloads and citations. Today, in Part II (of a three or four part series over two weeks), we present our findings (and detail our efforts in doing so) to see whether blogging improves the blogger’s reputation as part of our paper in progress. We are also asking you to tell us whether you think blogs influence economic policy and request that you provide us with anecdotes (or better evidence if you have it).
Does blogging improve reputation?
Davis et al. (2011) conducted a survey of academic economists in the U.S., with 299 (15%) responding. The survey asked these academics to list up to three living economists over the age of 60 and up to three under the age of 60 who they “regard with great respect, admiration, or reverence”. Gary Becker, Ken Arrow and Robert Solow were the top choices among the over 60s, and Paul Krugman, Gregory Mankiw and Daron Acemoglu the top choices among the under 60s. The under 60s list of 23 names contains a number of regular bloggers – in addition to Krugman and Mankiw are Steven Levitt, William Easterly, Nancy Folbre, Dani Rodrik, and Tyler Cowen.
We merge this list with a list of the top 500 economists according to the RePEc rankings (based on paper downloads, citations) and also code each of the RePEc top 500 according to whether they blog or not. This data is then used to estimate a probit model to see whether, conditional on RePEc ranking, individuals who blog are more likely to appear on the above list of favorite or admired economists. Table 2 shows the results, for the pooled sample in column 1, and separately for under 60 and over 60 economists in columns 2 and 3 respectively.
(here is the data and do file if you want to replicate or play around with this yourself)
In all three columns we see that, conditional on their RePEc rank, regular blogging is strongly and significantly associated with being more likely to be viewed as a favorite economist. Blogging has the same size impact as being in the top 50 of RePEc rankings for the under 60 economists, and a larger impact for the over 60 economists.
This evidence is thus consistent with the view that blogging helps build prestige and recognition in the profession, with bloggers being more likely to be admired or respected than other academics of similar (or in many cases better) publication records. This is of course only a correlation, and there are several caveats to consider. First, to the extent that blogging serves to increase the RePEc ratings by increasing own downloads (as seen in our previous post) and citations, the observed correlation will be a lower bound on the causal impact of blogging. However, if bloggers are also more likely to be engaged in other activities of a public intellectual, such as media appearances, writing books etc., and if these don’t all arise directly as a result of blogging, the estimates will conflate the impact of blogging with the impacts of these other activities, thereby overstating the impact of blogs. Nevertheless, given the large magnitude of the coefficient observed, it does not seem likely that all of the observed impact of blogging just reflects omitted variables, and therefore we view this evidence as strongly suggesting that blogging increases the influence, respect, or public image of the blogger.
Does blogging influence policy?
This is where we haven’t been able to find much evidence to date. A couple of days ago, Tyler Cowen discussed which intellectuals have influence, where he defined influence NOT as “influencing lots of other minds,” but as “changing the world.” He concluded that only a few intellectuals had influence defined this way and that “…it is very hard to have much influence.” In political science, Drezner and Farrell (2008) provide a number of anecdotes about how blog posts have helped bring down powerful politicians or had other policy effects.
We don’t know of many such cases in economics. The one clear example that comes to mind is the case of Kiva – a person-to-person microlending website- where a blog post by David Roodman explained that the way Kiva actually operated was different to the way that it implied it operated – leading to a story in the New York Times and modifications in the website. This is a case of economic bloggers as expert journalists – bringing facts to a new story. Another case is where Michael Clemens from the CGD and Gabriel Demombynes from the World Bank wrote a new paper and then blogged about it questioning the evaluation of Jeff Sachs’ Millennium Villages Project. A flurry of responses from the MVP camp (and responses to those) followed, there was some media attention, and a debate was scheduled than cancelled. It’s hard to ascribe the flurry to the blog rather than the paper, although it most likely helped fuel the ensuing debate. However, it is also hard to decide what, if anything changed as a result. Some minds were perhaps changed, which may influence similar cases in the future. But, is MVP changing how they are doing things as a result? We don’t know…
We would love to hear from readers, bloggers, and policy makers of other examples where blog posts have changed policy – particularly cases which have involved economic analysis, rather than just reporting.
Coming on Monday: Impact of Blogs Part III: Results from a new survey and an experiment!
Thanks Tim, these types of examples are exactly what we are looking for. Proving causality is going to be very difficult, but my sense is that:
i) very few posts actually influence policy
ii) there are very few readers of blogs who are actually in a position to influence policy,
but
iii) it only takes one post read by the right reader to potentially make a big difference. This poses enormous problems for statistical inference, since these are likely rare events, but I think it is still useful to see whether there are in fact any plausible candidates.
You ask for examples of blogging changing policy. A couple of very minor tales.....
I am told (but have no actual proof of) that my blog has produced two policies for the Liberal Democrat Party in the UK. Which is slightly odd as I'm not a member or supporter of that party.
But one of my readers is and as a result of my wittering on about the causes of the gender pay gap (it's, obviously, all about child care) the party has, urged on by my reader, adopted the policy that it should be transferable maternity/paternity leave, not maternity.
This now seems to be being implemented into actual government policy.
The second is as a result of the same reader: that the personal tax allowance in the UK should be raised to the same level as the full year full time minimum wage. I'm told that this will be suggested as policy in the near future.
I ca\n't prove any of this and it's all a rather roundabout route but I'm told that it is true and so pass it on with that caveat.
I loved Part I, but this is even better! :)
It's not written by an economist per se, but I can tell you that the Calculated Risk blog has had enormous policy impact in the US. Lots of policymakers and economists (and even bank analysts) didn't know that much about the details of the mortgage market (especially all the nitty gritty servicing issues) and got up to speed very quickly with Tanta's posts.
Tim,
Thanks - this is very nice. Drezner says that he was surprised to find out how many influential people read blogs - at least the elite ones. So, even the smaller, niche bloggers have a chance for influence if something they keep talking about gets picked up by the elite bloggers or a large media outlet read by the influential policy-maker.
Berk.
Don't you mean Bob Solow?
Before he wrote much more generally about macroeconomics for HBR, Umair Haque used to run a blog called Bubblegeneration which dealt primarily with digital media economics. Around 2005 he published a paper there called The New Economics of Media
(here's a copy http://www.scribd.com/doc/12177741/Media-Economics-The-New-Economics-of…-)
It would be little exaggeration to say that anyone making a plausible claim to understand media economics from that point on had necessarilly read it and altered their thinking accordingly. In a minor way it influenced policy at one company since my advice to the media business I worked for up to 2010 on which websites to buy or avoid, build or not build etc, was materially informed by this analysis. Anecdotally, it informed similar analyses at other media businesses.
Technically it is/was not a blog post - it is a presentation, which I am reasonably sure was originally published in powerpoint. It was however published on Umair's Bubblegeneration blog.
Yes, not the lesser known "Gary Solow" - now corrected.
Of course, what I actually want is the government to introduce that lovely large subsidy program for people who write blogs but I'm finding that one a bit of a hard sell.....
Asking the audience for instances of where a blog influences is self-selecting of those instances.
You could ask an open question: does it influence, yes or no, and, if so, give an example.
In noting increased downloads from an author who has a blog than from one who doesn't, you are failing to control for the composition of the audience.
Let's say an economist doesn't blog: he gets so many downloads, mostly from hits by other economists in his field.
Let's say an economist blogs, and has an audience composed of economists and many non-economists, and even students from his class, and he gets hits and download requests mostly from his students.
You could say the blogger got more hits. That's true. But, the audience is different and thus the comparison has to be adjusted.
Our point on this was simply that the RePec ranking of economists includes paper download statistics as one element of the ranking. So it doesn't matter who downloads the paper, if a blogger manages to boost his or her downloads by blogging a lot about their own papers, this would still increase their RePec rank - and therefore by controlling for RePec rank, we would be underestimating the overall effect of blogging.
David and Berk, this is very fun, but I think the reputation-blogging correlation is spurious. If you look at the high reputation bloggers -- e.g. Levitt, Rodrik, Becker, Mankiw, Krugman -- they already had a high public profile before blogging, through books for general audiences, textbook, newspaper columns, etc. (and a couple had those little things called Nobels). The incentive to blog depends on how big an audience you expect to attract so those who were already well known have a stronger incentive to start blogging. But these quibbles aside, I hope you continue this vein of research! best. Bill
PS I will let you know if my modest reputation collapses now that I have stopped blogging.
Thanks David and Berk for great evidence and discussion. The questions you raise are much bigger than blogging -- a compliment, not a critique. There would be at least as many unresolved empirical questions if the title of this post were "Journal articles enhance the authors' reputation. But do they influence policy?"
Lant Pritchett has argued, for example, that most of the economics of education literature has had little policy influence:
http://web.hks.harvard.edu/publications/citation.aspx?PubId=7119
I imagine there is related literature from other fields, literature of which I'm ignorant.
Measuring and attributing policy change caused by blogging faces many of the empirical challenges that measuring and attributing policy impact from any kind of writing faces. I'm sure it could be done, creatively, but it seems to me that very little has been done in this entire area, of which blogging is a special case.
Hi Bill,
Thanks for the comment - we recognize this is the hardest part of our exercise to go much beyond correlations, and certainly recognize this issue. One can certainly point to cases where it seems that people have built or enhanced their reputations through blogging also, but we recognize that causal links will be tough to make there.
That's why you should stay tuned for Monday, when we present experimental evidence that shows, at least for one blog, causal impacts on reputation.
A useful approach to finding out which info sources influence policymakers is to, well, ask them.
There is a bit of research on this topic (loosely referred to as "evidence to policy") in the US health services research field.
This paper
http://content.healthaffairs.org/content/21/2/264.full
shows the findings of a survey in 2000 of state level health policymakers about which info sources they use, how often, and what characteristics of an info source makes them more likely to turn to them. They prefer short (very short) info sources, written in an accessible style - non-jargony, and relevance to "issue at hand" is critical. I can't recall the rest. All in all, the characteristics policymakers look for in an info source sound remarkably blog-like (the survey was done too long ago for blogs to be relevant).
We have a cadre of folks in the World Bank who are in a sense our "policymakers" (operational staff involved with project preparation). Why not do a survey monkey survey and see what info sources they say they are turning too, and why? Could be interesting.
Thanks April,
As you will see in the next installment (on Monday), we did ask WB economists some of these questions in our survey on the use of this medium. The problem is that of all the groups we surveyed (using survey monkey indeed), they had by far the lowest response rate (at about 25%). Whereas graduate students, junior faculty, and development field workers responded at much much higher percentages. It seems that people in the Bank are bombarded with so many of these things that they tune most of it out.
Berk.
I'm sorry, though not surprised, to hear about the low response rate by WB economists.
I know I'm going a bit off the focus of measurement here - but I've always thought that value of blogs was opening up the conversations about research that takes place among groups of experts working on, or interested in, similar topics. The blogosphere lets interested but distant "colleagues" overhear the stimulating hallway conversation from afar - and even jump in when they have something to say.
By allowing more people to participate or even just listen to those conversations - I feel blogs help whole fields of researchers and practitioners move forward more quickly; and I believe that folks who live in peripheral areas/ institutions with low density of like-minded people benefit the most. I think this effect is less noticeable to folks like us who have long been situated in locations with large numbers of colleagues to interact with.
So I see the main goal of blogging and participating in conversations in the blogosphere as one of opening up our discussions to broader participation by far-off colleagues wrestling with the same or similar questions.
I expect that this broader inclusion in discussions will improve the quality of research papers also (though more on the periphery than in core areas). I see this as an externality and, alas, I have no suggestions at all for how to measure that influence.
Maybe you could have one blog post where you simply ask what people think is the point of blogging? I expect the answers would be quite varied. Maybe I'm further off the reservation than the rest of this blog's readers, but paper downloads never would have occurred to me as an indicator of impact.
Downloads and citations are one aspect of a variety of potential impacts of blogs. I don't think I would downplay the importance of dissemination of research, especially, for us, the policy-relevant kind.
But patience, more is coming on Monday, perhaps closer to the things you are thinking about...
Berk.
Bloggers among us would of course like to think that blogging has some influence on the opinions and actions of others. But finding the evidence of this is as you say tricky.
It seems quite likely that attribution of change to blogs would be quite hard to determine though since it's likely that you can write dozens of posts with little result and then get one picked up and used, or that the cumulative effect of something discussed on several blogs brings it to attention rather than one blogger. Similarly blogging might open the door to get invited to speak, or consult or advise on something which then has an impact.
Given the nature of this type of influence it's probably better to collect "stories" of impact from the point of view of both bloggers and policy makers rather than trying to do quantitative analysis. If you collect enough of these you will be able to get some "systematic anecdotal evidence" to tell you whether blogs do ever influence policy and if so how they do it, and how frequent this is, and what type of blog or blogger is more influential.
Would love to see the results since I've been trying to persuade the organization where I work we need to start blogging, and would like to be able to do more than just point to the Bank and say look those guys are doing it!
Gabriel Demombynes and John McArthur participated in an Millennium Villages Project debate, at the CSAE annual conference, March 2011.
Video here (fourth video in the player): http://www.csae.ox.ac.uk/conferences/2011-EdiA/video.html
Papers here: http://www.csae.ox.ac.uk/conferences/2011-EdiA/plenaries.html
Summary on pages 2 & 3 here: http://www.csae.ox.ac.uk/output/newsletters/pdfs/CSAE-newsletter-04-201…
How would you suggest measuring blggers effectiveness in influencing policy? what are the pre requisites? Who are readers of the blog? what is the affilitation of the blog? where is the blog hosted? how does new information posted on the blog gets informed to reader? Is registering with an email is a key? Is the topic a hot topic in the country/region/world?
I would suggest considering all of the above in deciding on the strength of the influence