This is the fifth year in which I have attempted to put together data on development journals that is not otherwise publicly available or easy to access (see 2017, 2018, 2019, 2020). I once again thank all the journal editors and editorial staff who graciously shared their statistics with me.
Measures of Journal Quality
The most well-known metric of journal quality is its impact factor. The standard impact factor is the mean number of citations in the last year of papers published in the journal in the past 2 years, while the 5-year is the mean number of cites in the last year of papers published in the last 5. I compliment these stats with RePec’s journal rankings which take into account article downloads and abstract views in addition to citations.
Note: impact factors reported to journals to me. I think the IZA Journal impact factor is either measured differently or heavily influenced by an outlier.
As in previous years, I note that the impact factors of 2 to 4 indicate very low numbers of citations, which reflect the long time lags for publishing in economics. Coupled with the fact that the mean can be influenced by one or two papers that gather many citations, I prefer comparing journals by examining the distribution of citations. I therefore took the 2019 issues of each journal, and extract in March 2020 the Google Scholar citations of each paper published. I also then manually selected the development papers from the AER and QJE, and from three next tier general journals (ReStat, EJ and AEJ Applied) for comparison.
Figure 1: Boxplot of citations as of March 2021 of articles published in 2019
Notes: Journals ordered by mean Google scholar citations. In contrast to the impact factors, I look at total lifetime citations for papers published in 2019.
A few remarks based on this figure:
· While these citations generally line up with the impact factors, the figure highlights the enormous variation in citations and the limits of comparing things based on means. Among the development journals, a regression of citations on journal fixed effects has a R-squared of only 0.07, and the standard deviation of citations is about the same magnitude as the mean for most development journals. Moreover, while the means and medians are much higher for the top general interest journals, six of the development journals (JDE, WD, EDCC, WBRO, JDS and DmtPolicyReview) all had at least one paper published in 2019 that already has at least 64 cites, the 25th percentile for development papers published in the AER and QJE. So don’t judge a paper just on the journal it is published in.
· I am sometimes asked by colleagues about the relative merits/rankings of AEJ Applied, ReStat and the Economic Journal (EJ): this figure shows similar median citations for development papers (in the 31 to 35 range), and a lot of overlap in distributions. However, AEJ Applied and the EJ published more development papers in 2019 than ReStat (15 and 19, vs 10).
· As always, these comparisons are complicated further by issues of differences in the ages of the papers by time they finally get published at different journals.
How many submissions are received and what are the chances of a paper getting accepted?
Table 2 shows the number of submissions, and number of papers published each year (excluding online supplements and proceedings), and the acceptance rate. The acceptance rate can be complicated due to articles being submitted one year and accepted in another, and I use what each journal office reports to me. You could also just take the number published over the average of the number of submissions in the last two years and get similar figures.
One of the advantages of collecting this information for five years now is that we can see trends and changes occurring over time. With the COVID-19 pandemic, I think many of us originally thought that there would be a reduction in papers written as researchers struggled with juggling home-schooling, learning how to teach online, and disruptions to their ability to do fieldwork. But instead there was a large increase in journal submissions: from 7644 in 2018 and 7458 in 2019 across all the development journals in this Table to an amazing 10,256 submissions in 2020!!! That reflects a 37% increase on 2019 submissions. In absolute terms this increase was most stark at World Development, which received an extra 1,596 submissions, a 58% increase. Many journals received 20% increases, and Economia and the Journal of Development Effectiveness both received over 75% increases in submissions.
Many of these submissions may still be in the review process, and so it is perhaps too soon to completely see whether the journals are expanding space as the supply of papers increases, or whether acceptance rates are dropping, but we see a bit of both. For example, World Development increased from 274 articles in 2019 to 410 articles in 2020, in part because of shorter paper and notes being included for special issues; EDCC increased the number of pages per issue to reduce the time between acceptance and print, going from 24-27 papers each year in 2016-2019 to 43 papers in 2020; and the JDE and Development Policy Review also saw increases in papers published. In contrast, we see fewer papers published in 2020 than 2019 at Economia and the Journal of African Economies, and their acceptance rates fell. Acceptance rates were under 10 percent at 6 of the development journals, and fell to under 5 percent at the WBER and Journal of African Economies.
Of course, the difficult thing we can’t say anything about from these statistics is whether the quality of the submissions changed, and, in particular, whether this growth in submissions comes from lower quality papers. Several journal editors have noted a rise in submissions that are well below the bar for publication, which, as we will see, shows up in part in high desk rejection rates.
How long does the review process take?
Table 3 provides information on decision times. The first column shows the desk rejection rate, which is above 50 percent at most journals, and up to 85 percent. I then use the acceptance rates from Table 2 together with these desk rejection rates to give the mean chance of acceptance conditional on the paper going to referees. These are a bit more encouraging than the rates above – and should be some guidance for your over-zealous referee 2’s who want to reject everything – if editors are doing a good job in desk rejecting, then at least one-quarter of the papers getting sent to referees should be good candidates for accepting. The remaining columns then report the amount of time taken for decisions. I asked for both unconditional on going to referees (which captures lots of quick desk rejects also), and conditional on going to referees.
I know many journals struggled with getting referees to agree to referee papers this year, and then with getting referees to respond on time. Several of them pledged efforts to improve these response times. That said, given the massive number of submissions, and the over 2,000 papers that needed to get refereed last year, the mean times to decision and percent refereed in 3 and 6 months don’t look so different from 2019. Conditional on going to referees, 3-4 months has been about average, and we see that still for many journals. The challenge is the long tails, and the chance as an author of ending up with more than 6 months or so until you hear back. Please do your part as a referee – say no quickly if you can’t do it, and try to referee within a month if you accept.
Progress towards open access
I thought I’d also check in this year on how the move towards open-access is going. Of course, many papers can be found on the authors’ websites and as working papers, but this just relates to formal open-access publishing at the journals. You can see that a couple of journals are fully open-access – the IZA Journal of Development and Migration, and Development Engineering. At the rest of the journals we are starting to see slightly more papers published open-access. One reason is through the read and publish deals signed by different countries or universities with publishers, which pay for their authors’ work to be made open-access – rather than from individual authors paying the $3,000-odd dollars. For example, the University of California recently signed such a deal with Elsevier. Indeed the IZA Journal of Development and Migration “explicitly does not expect any author to pay publication fees from her/his own funds. Nor do we charge any submission fees. Instead the IZA Open Access Journals aim at exploiting available funds at research institutions and third-party resources (e.g. external funding of research projects and publications).”.
Addendum: I should note that these are the "list price" open access charges. Some journals may offer lower rates for authors from developing countries. Moreover, some journals may also make online access free for readers from non-OECD countries. This is true of the World Bank journals. See https://academic.oup.com/wber/subscribe for details for the WBER and https://academic.oup.com/wbro/subscribe for details for the WBRO.
Other Development Journal News
Finally I asked journals if they had any news to report, or any other changes to note. A couple of things to note:
· New editors at Economia, Journal of Development Studies, and World Development: Gustavo Bobonis (University of Toronto) and Irene Brambilla (Universidad Nacional de la Plata) have taken over as editors at Economia; Chuan Liao (Arizona State) and Jampel Dell’Angelo (VU University, Amsterdam) are the new editors in chief at World Development; and at the Journal of Development Studies, the three new managing editors added are Jennifer Brass (Indiana, Political Science), Lisa Chauvet (Paris, Economics) and David Fielding (Manchester, Economics).
· Submission fee introduced at EDCC: after a record 567 submissions in 2020 (and an exhausted chief editor), they introduced a submission fee of $50 on January 1st 2021. The objective is to discourage frivolous submissions and reduce the editorial workload.
· Trials of new submission processes, paper options and revision processes:
o The Journal of Development Studies is trialing a new process for papers for which significant revisions would be required (at the margin of rejecting but where the paper has a good idea) by allowing authors to indicate to the editor how they would respond before going ahead with a revised version.
o The Journal of African Economies have introduced the option of considering papers with a full set of reports which have been rejected by another journal. Authors are asked to submit the decision letter from the editors and all reports with their responses to the comments. We noted last year that EDCC has also introduced this new optional prior review process, where if you have only sent the paper to top-5 journals, and then go straight to EDCC, they offer an expedited review where the paper might go to only one, or perhaps zero, referees.
o The Journal of Development Economics introduced a short paper track with a separate and faster workflow. Papers can be submitted to this track by the authors or, at the option of our coeditors and the permission of the authors, rerouted to this track. They continue to do registered reports although the number of submissions dropped, likely due to field work restrictions. The journal notes that their relatively flexible policy on revisions for phase I acceptances proved helpful given the need to modify in-field projects.
o Development Engineering introduced the development engineering co-mentorship program, that was put in place to increase publication rates for low and middle income country scholars, and provide more visibility and recognition for their work. Unfortunately, only a few authors have reached out about the program. They note a larger effort will be required to advertise the program and recommend the program to submitters who may qualify. The editors also wrote a piece on what types of papers they are looking for going forward and providing a guide to authors interested in publishing with the journal. Finally, they note two open calls for special issues:
§ Special Issue on Geospatial Analysis for Development: The SI seeks to promote geospatial data and related analytical tools in global development. To that end, we will publish manuscripts that describe and use cutting-edge tools, datasets, and applications of geospatial data.
§ Special Issue on Cost Estimation: The SI seeks to explore the use of cost estimation as a tool for resource-constrained design and the evaluation of products, services and innovations intended for marginalized populations–particularly in the areas of safe water, sanitation, energy, food, agriculture, education, and shelter (we note that healthcare costs were recently explored by other journals). Our aim is to elucidate best practices and identify trends, challenges, and opportunities.
Join the Conversation