Today we continue our discussion from Monday with Arug Agrawal, editor of World Development.
DI: World Development is more inter-disciplinary than many other development journals. This sometimes causes communication difficulties for authors when they strike referees from different disciplines who worry about different issues or don’t share an understanding of some of the same methodological approaches. Do you have any advice for authors sending papers to your journal on how to best deal with this issue?
AA: This is a real issue, and to some extent I think of disciplinary affiliation is an important, but only one proxy for communication difficulties. Communication difficulties can also arise owing to other kinds of differences: between subfields within a discipline, between substantive domains (say health vs. education vs. sanitation) in the same subfield, and even perhaps between researchers working in different regions.
We have a database of nearly 17,000 reviewers and we do our best to match reviewers to papers initially on the basis of keyword searches, and then by looking at the kind of work done by the individuals that match the keyword search. Typically, we try to have at least one reviewer who works on the region if the paper focuses on a single country or region, and another reviewer whose research overlaps the paper’s substantive concerns. We will also do our best not to assign a paper written by an economist to an anthropologist reviewer precisely for the reason you mention (although a reviewer is often the hardest on a paper that is close to his or her specialty! So yes, I worry a lot about a good match between a reviewer and a paper, and try to avoid obvious problems of “translation” across intellectual, disciplinary, or methodological formations.
If an author is particularly concerned about such an issue, it is always useful to highlight it in the cover letter. I will almost never send a paper to a reviewer suggested by an author, but I will also almost always take into account general concerns or preferences expressed by the author regarding types of reviewers.
DI: As a follow-up issue, do you typically try and assign a mix of referees from different disciplines to each paper, or is it common for papers by economists to typically get sent to economists as referees, and papers by sociologists to get sent to other sociologists, etc.?
AA: Unless a paper is evidently speaking to a topic on which there is a relatively robust interdisciplinary literature, I am careful not to assign reviewers who may not appreciate a paper’s arguments because of differences in disciplinary affiliation.
DI: Do you have a cut-off as to what constitutes a development paper? For example, would papers on Korea or Estonia be considered development papers? Or does it depend on the subject, so you would even consider certain papers based in the U.S. or Western Europe?
AA: Hmmm, I am not sure I fully appreciate this question. If it is about whether there is an income threshold beyond which a paper ceases to be a good fit for World Development, perhaps that is true, but I have not defined that threshold particularly clearly to myself. I think if a paper is about poverty in Western Europe or the United States or Japan, it could make a claim to why it is a good fit for World Development. But frankly, we get very few papers like that. If it is a good paper, I would certainly consider it.
DI: As far as we are aware, World Development currently does not have a data replication policy (where authors of accepted papers have to submit their data and code so that anyone who wants to can replicate the tables in the paper). Is this correct? Is this something you are thinking about requiring for accepted papers, or not something you see as a priority?
AA: You are right – we do not have a data replication policy. It is something I am actively considering and is a topic for our annual editorial team meeting for the year. I can’t say that it is the first priority for this year’s meeting, but it is something on which I would like us to develop a policy within the next two years, in consultation with editors of other development journals as well.
DI: Finally, our blog focuses on issues related to impact evaluation and measurement. Are there certain areas in these topics where you think World Development offers a home for work that might not be such a good fit at some other development journals? For example, do you think you are more likely to publish mixed methods approaches to evaluation? Would you publish pure qualitative evaluation work? On the measurement side, our view is that the profession would benefit from more good papers on improving measurement of development issues. Are you interested in papers that are purely about how to measure some aspect of development better?
AA: I don’t think we are more likely to publish papers that use mixed methods, or rely on a single evaluation approach/method, or because they are based purely on qualitative methods. Ultimately, what a paper says in answer to the “so what” question and how well it provides the answer is crucial to the publication decision.
But it may be true that if other development journals catering mostly or only to economists publish only quantitative or modeling or mathematically driven work, World Development is one of the few places where alternative ways of producing insights about development will continue to be valued.
Join the Conversation