In 2009 the American Economic Association launched four new journals. Over the past five years, the
American Economic Journal Applied Economics (AEJ Applied), edited by development economist Esther Duflo, has published a number of development papers related to impact evaluations. However, because of the relative newness of these journals, there appears to be some uncertainty about just how good this journal is: I have heard this for people coming up for tenure; for young researchers unsure about where to submit their paper; and it is also the case at the research department at the World Bank, where the AEJ applied is classified in the second tier of journals for the purpose of annual performance reviews, whereas what I tend to view as similar level (Review of Economics and Statistics (ReStat), Economic Journal (EJ), and Journal of the European Economic Association (JEEA)) are classified as first tier, as is the Journal of Development Economics (JDE).
However, I believe 5 years is now long enough for us to get a decent understanding of how good this journal is, and how it compares to the 122 year old EJ, 96 year old ReStat, and 10 year old JEEA. To do this, I looked at all issues of these journals for the years 2009, 2010 and 2011 and listed the empirical development papers they had published during this period. I did the same for the AEJ applied, and for comparison, also the AEJ Economic Policy (which has published a small number of development papers). For each development paper, I used Google Scholar to get the number of citations for the paper. To get a comparison to the JDE, I took the January issue each of these three years of the JDE, and got the Google Scholar citation stats for each paper in these issues as well.
Update: here is the Excel spreadsheet with this information.
Why these journals, and how much development do they publish?
Economists typically view the QJE, AER, JPE, Econometrica, and ReStud as the top five journals in Economics. As my colleagues Jishnu and Toan have shown, papers on the United States are far more likely to get published in these top five journals than papers on developing countries. Authors of development papers that don’t make a top-5 then typically try one of the next tier of highly respected general journals – ReStat, EJ, JEEA, AEJ, or otherwise go to a development journal like the JDE.
Figure 1 shows the number of empirical development papers published per year in these journals over the 2009-2011 period. We see that ReStat and AEJ Applied publish the most development papers from among this group – an average of 15 a year in AEJ Applied and 14 a year in ReStat – while the EJ, JEEA, and AEJ Economic Policy publish very few empirical development papers.
How do citations compare for development articles across these journals?
Figure 2 shows the median number of citations in Google Scholar as of 12 June 2013 for these development papers, by year of publication. The median number of citations for articles published in 2009 in the AEJ applied currently stands at 70, compared to 73 for the JEEA, 81 for ReStat, and 117 for the EJ. Recall the sample size is very small for the EJ and JEEA. By way of comparison, papers in the January 2009 issue of the JDE had a median of only 31 citations.
The AEJ applied looks even better when it comes to papers published in 2010 and 2011, where it is equal top in 2010, and second in 2011 in terms of citations among these 6 journals.
AEJ Applied articles sit around less before publication
The article processing process is faster at AEJ applied and AEJ Economic Policy than at ReStat, EJ and JEEA. To start with, these journals referee papers faster, and appear less likely to send revisions back to referees before making a decision. Moreover, perhaps in part due to their relative newness, they do not have large publication backlogs, so once accepted, articles tend to appear in print within a few months. In contrast, ReStat has about a year lag between acceptance and publication. To examine the cumulative result of this, I looked in Google Scholar to find the first working paper version of each 2009 accepted paper, and thus measured the time between the paper appearing as a working paper, and the time it appeared in print – the longer this time period, the more time the article has to start gathering citations before publication. Figure 3 shows the result – the median published paper in the AEJ applied and AEJ Economic Policy has been around about 1.5 years less than the average paper in ReStat, EJ, or JEEA.
Thus if we wanted to correct for this, we should be comparing the citations of 2009 AEJ Applied papers to those of 2010 (or 2011) ReStat, JEEA, or EJ papers. From Figure 2 one sees that on this basis, AEJ applied papers would dominate the other journals in terms of citations.
What do I conclude from this?
Apart from concluding that I am very good at finding ways to distract myself from other work, this exercise suggests to me that Tenure Committees, Review boards, and the World Bank research department should view the AEJ applied as at least an equivalent to ReStat, the JEEA and EJ. Of course citation counts are only one measure of the importance/worth/impact of an article, but I am unaware of other metrics that would suggest a reason to believe these other journals are better. Personally I view these journals as equivalents when deciding where to submit my papers, and I have had a paper rejected at ReStat accepted at AEJ applied, and one rejected at AEJ applied accepted at ReStat. Given the shortage in journal space and the large number of excellent development papers being produced, the presence of the AEJ applied is a very welcome addition to the journal roster, and one that is playing an important role in disseminating impact evaluation work in particular.
However, I believe 5 years is now long enough for us to get a decent understanding of how good this journal is, and how it compares to the 122 year old EJ, 96 year old ReStat, and 10 year old JEEA. To do this, I looked at all issues of these journals for the years 2009, 2010 and 2011 and listed the empirical development papers they had published during this period. I did the same for the AEJ applied, and for comparison, also the AEJ Economic Policy (which has published a small number of development papers). For each development paper, I used Google Scholar to get the number of citations for the paper. To get a comparison to the JDE, I took the January issue each of these three years of the JDE, and got the Google Scholar citation stats for each paper in these issues as well.
Update: here is the Excel spreadsheet with this information.
Why these journals, and how much development do they publish?
Economists typically view the QJE, AER, JPE, Econometrica, and ReStud as the top five journals in Economics. As my colleagues Jishnu and Toan have shown, papers on the United States are far more likely to get published in these top five journals than papers on developing countries. Authors of development papers that don’t make a top-5 then typically try one of the next tier of highly respected general journals – ReStat, EJ, JEEA, AEJ, or otherwise go to a development journal like the JDE.
Figure 1 shows the number of empirical development papers published per year in these journals over the 2009-2011 period. We see that ReStat and AEJ Applied publish the most development papers from among this group – an average of 15 a year in AEJ Applied and 14 a year in ReStat – while the EJ, JEEA, and AEJ Economic Policy publish very few empirical development papers.
How do citations compare for development articles across these journals?
Figure 2 shows the median number of citations in Google Scholar as of 12 June 2013 for these development papers, by year of publication. The median number of citations for articles published in 2009 in the AEJ applied currently stands at 70, compared to 73 for the JEEA, 81 for ReStat, and 117 for the EJ. Recall the sample size is very small for the EJ and JEEA. By way of comparison, papers in the January 2009 issue of the JDE had a median of only 31 citations.
The AEJ applied looks even better when it comes to papers published in 2010 and 2011, where it is equal top in 2010, and second in 2011 in terms of citations among these 6 journals.
AEJ Applied articles sit around less before publication
The article processing process is faster at AEJ applied and AEJ Economic Policy than at ReStat, EJ and JEEA. To start with, these journals referee papers faster, and appear less likely to send revisions back to referees before making a decision. Moreover, perhaps in part due to their relative newness, they do not have large publication backlogs, so once accepted, articles tend to appear in print within a few months. In contrast, ReStat has about a year lag between acceptance and publication. To examine the cumulative result of this, I looked in Google Scholar to find the first working paper version of each 2009 accepted paper, and thus measured the time between the paper appearing as a working paper, and the time it appeared in print – the longer this time period, the more time the article has to start gathering citations before publication. Figure 3 shows the result – the median published paper in the AEJ applied and AEJ Economic Policy has been around about 1.5 years less than the average paper in ReStat, EJ, or JEEA.
Thus if we wanted to correct for this, we should be comparing the citations of 2009 AEJ Applied papers to those of 2010 (or 2011) ReStat, JEEA, or EJ papers. From Figure 2 one sees that on this basis, AEJ applied papers would dominate the other journals in terms of citations.
What do I conclude from this?
Apart from concluding that I am very good at finding ways to distract myself from other work, this exercise suggests to me that Tenure Committees, Review boards, and the World Bank research department should view the AEJ applied as at least an equivalent to ReStat, the JEEA and EJ. Of course citation counts are only one measure of the importance/worth/impact of an article, but I am unaware of other metrics that would suggest a reason to believe these other journals are better. Personally I view these journals as equivalents when deciding where to submit my papers, and I have had a paper rejected at ReStat accepted at AEJ applied, and one rejected at AEJ applied accepted at ReStat. Given the shortage in journal space and the large number of excellent development papers being produced, the presence of the AEJ applied is a very welcome addition to the journal roster, and one that is playing an important role in disseminating impact evaluation work in particular.
Join the Conversation