This is the fourth year in which I have attempted to put together data on development journals that is not otherwise publicly available or easy to access (see 2017, 2018, 2019). I once again thank all the journal editors and editorial staff who graciously shared their statistics with me, and this year additionally thank Ben Ewing who wrote code to extract all the Google Scholar citation data.
1. Measures of Journal Quality
The most well-known metric of journal quality is its impact factor. The standard impact factor is the mean number of citations in the last year of papers published in the journal in the past 2 years, while the 5-year is the mean number of cites in the last year of papers published in the last 5. I compliment these stats with RePec’s journal rankings which take into account article downloads and abstract views in addition to citations.
As in previous years, I note that the impact factors of 2 to 4 indicate very low numbers of citations, which reflect the long time lags for publishing in economics. Coupled with the fact that the mean can be influenced by one or two papers that gather many citations, I prefer comparing journals by examining the distribution of citations. I therefore had Ben take the 2018 issues of each journal, and extract in March 2020 the Google Scholar citations of each paper published. I also then manually selected the development papers from the AER and from three next tier general journals (ReStat, EJ and AEJ Applied) for comparison. I also added the Journal of International Economics (JIE) given the strong overlap between trade and development.
Figure 1: Boxplot of citations as of March 2020 of articles published in 2018
Notes: Journals ordered by mean Google scholar citations. One outlier from the AER with more than 1,000 citations is omitted so as to show the spread clearly among other journals. In contrast to the impact factors, I look at total lifetime citations for papers published in 2018.
A few remarks based on this figure:
· I am sometimes asked by colleagues about the relative merits/rankings of AEJ Applied, ReStat and the Economic Journal (EJ): this figure shows similar median citations for development papers (in the 26 to 33 range), and a lot of overlap in distributions. However, AEJ Applied and the EJ both published at least 25 development papers in 2018, compared to only 7 in ReStat.
· In terms of development journals, there is no clear single top journal, but the group of the JIE, JDE, WBER and WBRO separate themselves somewhat as the top group, with mean citation rates all in the 19-26 range and medians in the 14-22 range. But we see a top tail of well-cited papers in many journals – don’t judge the paper by the journal.
· As always, these comparisons are complicated further by issues of differences in the ages of the papers by time they finally get published at different journals. The extreme case is the point I left off the figure – Dave Donaldson’s Railroads of the Raj paper, which first appeared as an NBER working paper in 2010, and was published in the 2018 AER with over 1,100 citations now.
2 What are the chances of a paper getting accepted?
Table 2 shows the number of submissions, and number of papers published each year (excluding online supplements and proceedings), and the acceptance rate. The acceptance rate can be complicated due to articles being submitted one year and accepted in another, and I use what each journal office reports to me. You could also just take the number published over the average of the number of submissions in the last two years and get similar figures.
Notes: (a)-(f) indicate issues to do with how special issues are accounted for, as well as editor estimates.
My main take-aways from this table:
· Last year’s blog noted the tremendous growth in submissions that journals have received over time. The last year appears to show a levelling off in the number of submissions at many journals – but still around 7,500 papers submitted in 2019!!!
· Acceptance rates are hard for journals to track because of papers being in different stages of the process, but comparing the four years in Table 2B shows acceptance rates have been fairly stable over the last four years at journals that receive a lot of submissions, but bounce around a bit in journals that get less than 100 submissions per year.
3. How long does the review process take?
Table 3 provides information on decision times. The first column shows the desk rejection rate, which is above 50 percent at most journals, and up to 83 percent. I then use the acceptance rates from Table 2 together with these desk rejection rates to give the mean chance of acceptance conditional on the paper going to referees. These are a bit more encouraging than the rates above – and should be some guidance for your over-zealous referee 2’s who want to reject everything – if editors are doing a good job in desk rejecting, then at least one-third of the papers getting sent to referees should be good candidates for accepting. The remaining columns then report the amount of time taken for decisions. I asked for both unconditional on going to referees (which captures lots of quick desk rejects also), and conditional on going to referees. However, not all journals are able to split their data this way, and I caution that sometimes it is unclear whether the last two columns are conditional on being refereed, or also include all the desk rejects.
My takeaway from this is that, conditional on going to referees, 3-4 months is about average. However, this may change with current conditions. Several editors have just posted in the last few days that they are getting a sudden surge in submissions, but at the same time getting more referee requests than usual turning down requests. Larry Katz tweeted that QJE submissions surged to record levels the past month, and in the replies, Seema Jayachandran notes that she is seeing slower referee reports and more declined requests. This is not just the case for economics, this tweet notes that an editor at the American Journal of Political Science saw a 27% increase in submissions and 54% decline in review invitations accepted since Covid-19 broke out (albeit this was only 12 more submissions, so take numbers with a grain of salt).
So please do your part if you can, and if you are going to submit a paper at the moment, be extra patient.
4. How common are multiple rounds of revisions, and for papers to get rejected after R&Rs?
Each year, in addition to the first three categories, I choose one additional aspect of publishing to look at – past years have looked at representation of who is getting published, and on open access and transparency policies. This year I thought I’d ask about how editors deal with revisions, and whether papers typically go through multiple rounds, whether they get sent back to referees after revisions, and what share of those getting an R&R ultimately get rejected. Since this was not systematically collected in the information systems of journals, I will instead share responses here for some journals:
· Development Engineering: Papers are usually sent back to reviewers if major revisions were requested, and sometimes when minor revisions were requested if the Editor wants a second opinion.
· Development Policy Review: Papers are first assessed by editors before going to referees, and only those that are close to being publishable are sent to referees. As a result, referees usually are favorable with only minor corrections suggested, and perhaps 5% of papers sent for review undergo more than one round of corrections.
· Economia-Lacea: papers given an R&R are rarely rejected (in the last year none were). If the author has properly addressed the comments, the editor will make a decision on the first revision without sending to referees.
· Economic Development and Cultural Change: the editor notes that his initial goal was to have a maximum of one major revision and one minor revision/conditional acceptance. But in practice, there is often two or even three rounds of major revisions – this usually comes from young authors who are inexperienced dealing with referees, and need a lot of help and information to get the paper to required standard. The editor’s supplied this nice little table showing decisions by round:
· IZA Journal of Development and Migration: The journal aims to make up or down decisions on the first round, and mostly done this. Second revisions are rare, and the editor does not think they have ever done a third revision.
· Journal of African Economies: 57% of first revisions have a second revision asked, although none were sent for a third revision. In most cases first revisions are sent back to referees.
· Journal of Development Economics: The editor typically sends back first revisions to referees. Approximately 5% of first revisions are rejected; 25% of first revisions are sent for a second revision; and 3 papers last year had a third revision requested.
· Journal of Development Studies: 10% of first revisions are rejected; 75% are sent for a second revision (often very minor or editorial), and 8% go to a third revision (usually editorial). Revisions are usually sent back to referees unless the referees had few comments or already were very supportive.
· World Bank Economic Review: note that I am a co-editor here, and I typically try to decide on revisions without sending papers back to referees, but other editors may do differently. Conditional on an R&R, about 11 percent of papers get rejected. Half of accepted papers were accepted after the first revision; for those going to a second revision, only one paper was rejected, one was sent for a third revision, and 89% were accepted.
· World Development: revisions are sent back to reviewers if the first round asked for major revisions, or if there was a first round conflict between reviewer recommendations. If the referees recommend minor revisions, the editor usually decides without sending back to referees. 19% of first revisions are rejected.
5. Other Development Journal News
Finally I asked journals if they had any news to report, or any big changes to note. A couple of things to note:
· Berber Kramer, the new editor of Development Engineering, wanted to clarify that the scope of the journal extends more broadly than just applications of technology and engineering to development. They note they consider engineering more broadly, including behavioral interventions that change (or “engineer”) one’s decision-making environment, so studies on choice architecture etc. are also welcome.
· EDCC has a new optional prior review process, where if you have only sent the paper to top-5 journals, and then go straight to EDCC, they offer an expedited review where the paper might go to only one, or perhaps zero, referees. This was introduced in 2018, although no submission has tried out the process yet.
· Finally, the end of page numbers in referencing some articles: both the Journal of Development Economics and World Development have moved from page numbers to Article numbering.
Join the Conversation