Syndicate content

Do the Bank’s Operational Staff Support the Bank’s Research Department?

Martin Ravallion's picture

The quality of development projects depends in part on how well grounded project preparation is in knowledge about what works and what does not. Development practitioners need to be well informed if their projects are to have impact.

The World Bank’s in-house research department—the Development Research Group (DECRG)—is the main unit aiming to supply relevant research findings to Bank operations, as well as to external clients. It is not a large department, accounting for about 1% of the Bank’s administrative budget. But it produces the majority of the Bank’s research, and has a high profile internationally. Indeed, it is often ranked ahead of almost all universities and think tanks in development economics, measured by the quantity of research outputs, downloads and citations to research findings. For example, the highly-regarded and much-watched ranking done by the IDEAS project currently puts DECRG ahead of all but one university. 


But there are differing views within the Bank about DECRG’s usefulness to the Bank’s operations. Anecdotes abound, and everyone has their own opinion. But what do the Bank’s staff members in operations say when asked in a strictly confidential survey? A recent survey of senior Bank staff by DEC asked: 

“Which of the following statements best represents your views about research at the Bank: (1) I would find it more valuable and useful if the institution were to out-source more of its research; (2) I would find it more valuable and useful if the institution committed more resources to research within an in-house research department.”

Over the whole sample of over 500 staff members, 63% supported the second option—for more resources for the research department. It might be conjectured that, across units of the Bank, those where support for the research department is stronger tended to respond to the survey at a higher rate. But this is not borne out in the data. Indeed, on re-weighting the data to eliminate the potential bias due to selective response across units of the Bank I still find that 63% of staff supported more resources for the in-house research department.
 

Does familiarity come with more support or less?

The survey also asked:

“How familiar would you say you are with WB research products/services on a scale from 1 to 10 where 1 means not familiar at all and 10 means extremely familiar?”
The mean response was a 6 on the 10-point scale (which was also the mode and median).

The data reveal that those who are more familiar with Bank research findings tend to be more supportive of devoting more resources to the in-house research department. The following table shows this. It also gives the average responses to another question in the survey, on whether the respondent believes that the Bank’s research is “worth the investment.” Both show that support comes with familiarity. Only those who (by their own admission) know very little about the Bank’s research tend not to support funding it.
 

Table: Support for Bank research is greater amongst those more familiar with that research 

 

Familiarity with World Bank research

Percentage that supports more resources for the in-house research department

Percentage that believes that the Bank’s research is “worth the investment”

1 + 2 (low)*

50.0

5.6

3

34.0

25.0

4

51.4

48.0

5

63.2

43.9

6

63.5

62.3

7

61.3

71.4

8

79.7

81.8

9

88.9

78.3

10 (high)

82.1

62.1

Total

63.0

56.8

* Familiarity levels 1 and 2 aggregated because of the small number of respondents giving “1.”

One should be careful about giving this correlation a causal interpretation. Maybe greater familiarity leads to stronger support, or maybe it is the support that encourages familiarity. We can’t know. But it does make one wonder whether those operational staff who do not support more resources for research are well informed about that research.

When one uses a more sophisticated statistical model to try to explain how support for the in-house research department varies across the sample, a number of other interesting findings emerge (though the caveat on their causal interpretation remains):

  1. Support is significantly higher among staff mapped to the Bank’s “Economic Policy” sector, within its Poverty Reduction and Economic Management cluster. This is an area where the research department is clearly very strong.
  2. Possibly more surprising is that support is also significantly higher amongst staff mapped to the “Urbanization” sector in the Sustainable Development Network. This is not an area where the research department is very strong at present in terms of the number of researchers specializing on the topic, but it is clear that operational staff in this sector want more in-house research support.
  3. Amongst the Bank’s regional units, the greatest support (controlling for other factors) is from the Africa region.
  4. Support for greater resources for the research department is also greater among those who highly value the technical quality and policy relevance of the Bank’s research, those who think it helps with the Bank’s analytic work, and those who think it helps improve project quality of lending operations at entry.
  5. It also tends to be greater among those with a less favorable view of external consultants.
  6. Support does not vary with length of the staff member’s service in the Bank, and is no different on average between Bank staff based in Washington and those in country offices.


As the director of DECRG I can’t help but be pleased with the amount of support we have amongst Bank operational staff. Of course, the survey respondents were not asked if they would support extra resources for the research department at the expense of their own department. There I am sure we would not have fared so well. But this survey does at least cast doubt on any view that the Bank’s operational staff would rather see research out-sourced. That is a minority view.

There were many other interesting findings from this survey. My analysis of the data throws some new light on how much stronger incentives for learning within a large organization such as the World Bank translate into greater knowledge on the part of practitioners. And it reveals some of the frictions in how knowledge is diffused within the organization. You can read about these and other lessons from the survey in my paper “Knowledgable Bankers? The Demand for Research in World Bank Operations.”

 

Comments

Submitted by Anonymous on
Did you examine the possibility of a self-selection bias in those who responded? Could it be that respondents from the universe that finds DEC valuable were more likely to respond to the survey than those who find DEC marginal to their work? In the work I do - we use a lot of research - but it is applied research, fundamentally different from the kind of research that DEC researchers engage in. That we do not find any research that DEC does to be relevant for sectoral needs should not in itself be surprising -- the Bank has many sectors and each sectors, particularly those that are the most established - have evolving needs that are ALSO quite heterogeneous across regions. It may not be the most appropriate for DEC to acquire research personnel that focus on very specific areas. In places like LAC and EAP the development landscape has changed substantially over the last two decades...countries have become significantly richer, the role of the BaNK HAS CHANGED, AND THE WORLD HAS CHANGED - the problems that dominate today are quite different from the problems of some time ago... It would seem that these circumstances would call for outsourcing research -- so that re can find and identify research that remains most relevant to chainging needs -- rather than being stuck with (perhaps good) researchers interested in problems of the past.

Let me address the four main points you make. First, I did note the possibility of bias in my blog post, and it is discussed further in my paper, “Knowledgeable Bankers?” (http://bit.ly/wEY3bS) As reported in my post, when I correct for selective reporting across Bank units (some more supporting of research than others) the average level of support for greater resources for the research department still comes to 63%. The bias is negligible. However, we can’t correct for hidden sources of bias within Bank units. No survey can do that. Second, almost all DEC research is applied. It is quite rare for us to do theory. So I don’t know that the research you do is so fundamentally different. Of course, we strive for rigor and objectivity. And sometimes we use advanced analytic methods, as appropriate to the problem at hand. But the methods are applied to real world problems. Third, I am sorry that you don’t find our research relevant to your needs. We try to cover all sectors, but we are a relatively small unit (1% of Bank staff) so we naturally can’t cover everything. We try to prioritize. That is to be expected, as you note. It sounds like your sector of interest is not one we have been focusing on lately. Of course, the supply of research responds to demand, albeit with a lag; otherwise we could not stay in business! Possibly there has not been much demand for our help coming from your sector. But thankfully your view appears to be a minority one amongst the Bank’s operational staff. Fourth, it is not fair to say that we are only studying problems of the past. Naturally there are lags in research, though that is also true of research done outside the Bank. However, the bulk of our research relates to current problems and risks facing our client countries. And this focus is not evident in the bulk of academic research on development. I would encourage you to peruse our recent working papers (http://bit.ly/2FmZ73) or our monthly research newsletter (http://bit.ly/xz4JwL) with 38,000 subscribers. (Sign up if you would like to become one.) Or just visit our web site (http://bit.ly/eC9q86). You will see lots of new research on (for example) food and financial crises—their impacts, and how to respond to them. You will also see that we directly tie some of our research to current operational work, such as in the many impact evaluations we are involved in. (Visit our Development Impact blog at http://blogs.worldbank.org/impactevaluations.) We also try to anticipate issues that might merit future operational engagement. For example, the large amount of research we did on conditional cash transfers (http://bit.ly/x17CwG)helped encourage many countries to develop these schemes as a key element of their safety nets over the last two years, often with WB support. Please look at what we actually do, and tell me if you still think our research is not applied, not “relevant for sectoral needs” (as a generalization), or that we are only interested in “problems of the past.”