Measuring economic journals’ citation efficiency: a data envelopment analysis approach

This paper by using data envelopment analysis (DEA) and statistical inference evaluates the citation performance of 229 economic journals. The paper categorizes the journals into four main categories (A–D) based on their efficiency levels. The results are then compared to the 27 “core economic journals” as introduced by Diamond (Curr Contents 21(1):4–11, 1989). The results reveal that after more than 20 years Diamonds’ list of “core economic journals” is still valid. Finally, for the first time the paper uses data from four well-known databases (SSCI, Scopus, RePEc, Econlit) and two quality ranking reports (Kiel Institute internals ranking and ABS quality ranking report) in a DEA setting and in order to derive the ranking of 229 economic journals. The ten economic journals with the highest citation performance are Journal of Political Economy, Econometrica, Quarterly Journal of Economics, Journal of Financial Economics, Journal of Economic Literature, American Economic Review, Review of Economic Studies, Journal of Econometrics, Journal of Finance, Brookings Papers on Economic Activity.

[1]  Henk F. Moed,et al.  Measuring contextual citation impact of scientific journals , 2009, J. Informetrics.

[2]  Charles Oppenheim,et al.  Comparing alternatives to the Web of Science for coverage of the social sciences' literature , 2007, J. Informetrics.

[3]  F. Førsund,et al.  On the Origins of Data Envelopment Analysis , 2000 .

[4]  Vicente P. Guerrero-Bote,et al.  Journal maps on the basis of Scopus data: A comparison with the Journal Citation Reports of the ISI , 2010 .

[5]  Kevin W. Boyack,et al.  Toward a consensus map of science , 2009 .

[6]  Kevin W. Boyack,et al.  Toward a consensus map of science , 2009, J. Assoc. Inf. Sci. Technol..

[7]  森谷 正規,et al.  JOURNAL OF Policy Analysis and Management , 2021, Journal of Policy Analysis and Management.

[8]  Gabriel Pinski,et al.  Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics , 1976, Inf. Process. Manag..

[9]  R. Shepherd.  Theory of cost and production functions , 1970 .

[10]  Oscar Volij,et al.  The Measurement of Intellectual Influence , 2002 .

[11]  Massimo Franceschet,et al.  A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar , 2010, Scientometrics.

[12]  Nisa Bakkalbasi,et al.  An Examination of Citation Counts in a New Scholarly Communication Environment , 2005, D Lib Mag..

[13]  E. Garfield.  Citation indexes for science. A new dimension in documentation through association of ideas. 1955. , 1955, International journal of epidemiology.

[14]  Verein für Socialpolitik.  German economic review , 2000 .

[15]  David Hinkley,et al.  Bootstrap Methods: Another Look at the Jackknife , 2008 .

[16]  Abraham Charnes,et al.  Measuring the efficiency of decision making units , 1978 .

[17]  Pantelis Kalaitzidakis,et al.  RANKINGS OF ACADEMIC JOURNALS AND INSTITUTIONS IN ECONOMICS , 2003 .

[18]  Nickolaos G. Tzeremes,et al.  The effect of foreign ownership on SMEs performance: An efficiency analysis perspective , 2010 .

[19]  M. Farrell.  The Measurement of Productive Efficiency , 1957 .

[20]  C. Lovell,et al.  A Comparison of Alternative Approaches to the Measurement of Productive Efficiency , 1988 .

[21]  F. Pujol.  Ranking Journals Following a Matching Model Approach: An Application to Public Economics Journals , 2008 .

[22]  Cinzia Daraio,et al.  The differentiation of the strategic profile of higher education institutions. New positioning indicators based on microdata , 2008, Scientometrics.

[23]  Leâ Opold Simar,et al.  A general methodology for bootstrapping in non-parametric frontier models , 2000 .

[24]  D. Laband,et al.  The Relative Impacts of Economics Journals: 1970-1990 , 1994 .

[25]  W. Cook,et al.  Aggregating Incomplete Lists of Journal Rankings: An Application to Academic Accounting Journals* , 2010 .

[26]  Aidan Kelly,et al.  The Association of Business Schools: Academic journal quality guide, version 3 , 2009 .

[27]  O. H. Brownlee,et al.  ACTIVITY ANALYSIS OF PRODUCTION AND ALLOCATION , 1952 .

[28]  E. Garfield.  The history and meaning of the journal impact factor. , 2006, JAMA.

[29]  J. N. Boles.  The 1130 Farrell efficiency system-multiple products, multiple factors. , 1971 .

[30]  Goio Etxebarria,et al.  Use of Scopus and Google Scholar to measure social sciences production in four major Spanish universities , 2009, Scientometrics.

[31]  Eugene Garfield,et al.  Citation indexing - its theory and application in science, technology, and humanities , 1979 .

[32]  Klaus Ritzberger,et al.  A Ranking of Journals in Economics and Related Fields , 2008 .

[33]  Gaines H. Liner,et al.  Methods of ranking economics journals , 2004 .

[34]  Mike Thelwall,et al.  Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines , 2008, Scientometrics.

[35]  Nickolaos G. Tzeremes,et al.  International Competitiveness in the ICT Industry: Evaluating the Performance of the Top 50 Companies , 2007 .

[36]  Boaz Golany,et al.  Creating a consensus ranking of proposals from reviewers' partial ordinal rankings , 2007, Comput. Oper. Res..

[37]  D. Preßmar,et al.  Operations research proceedings , 1990 .

[38]  Henk F. Moed,et al.  Coverage and citation impact of oncological journals in the Web of Science and Scopus , 2008, J. Informetrics.

[39]  Henk F. Moed,et al.  Journal impact measures in bibliometric research , 2004, Scientometrics.

[40]  Lei Wang,et al.  Three options for citation tracking: Google Scholar, Scopus and Web of Science , 2006, Biomedical digital libraries.

[41]  Friedrich Schneider,et al.  The 2008 GEA Journal-Ranking for the Economics Profession , 2008 .

[42]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007 .

[43]  Pantelis Kalaitzidakis,et al.  Rankings of Academic Journals and Institutions , 2003 .

[44]  Mark D. Miller,et al.  Examining differences across journal rankings , 2005, CACM.

[45]  A. Charnes,et al.  Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis , 1984 .

[46]  S. Liebowitz,et al.  Assessing the Relative Impacts of Economic Journals , 1984 .

[47]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[48]  J. Burnham.  Scopus database: a review , 2006, Biomedical digital libraries.

[49]  Alireza Noruzi.  Google Scholar: The New Generation of Citation Indexes , 2005 .

[50]  G. Debreu.  The Coefficient of Resource Utilization , 1951 .

[51]  E GARFIELD,et al.  Citation indexes for science; a new dimension in documentation through association of ideas. , 2006, Science.

[52]  Alexander I. Pudovkin,et al.  Rank-normalized impact factor: A way to compare journal performance across subject categories , 2005, ASIST.

[53]  Kurt Hornik,et al.  Journal Ratings and Their Consensus Ranking , 2009 .

[54]  Léopold Simar,et al.  Advanced indicators of productivity of universitiesAn application of robust nonparametric methods to Italian data , 2006, Scientometrics.

[55]  Michel Zitt,et al.  Modifying the journal impact factor by fractional citation weighting: The audience factor , 2008, J. Assoc. Inf. Sci. Technol..

[56]  Timothy Coelli,et al.  An Introduction to Efficiency and Productivity Analysis , 1997 .

[57]  P. W. Wilson,et al.  Sensitivity Analysis of Efficiency Scores: How to Bootstrap in Nonparametric Frontier Models , 1998 .

[58]  P. W. Wilson,et al.  Statistical Inference in Nonparametric Frontier Models: The State of the Art , 1999 .

[59]  Sergio Perelman,et al.  A comparison of parametric and non-parametric distance functions: With application to European railways , 1999, Eur. J. Oper. Res..

[60]  P. W. Wilson,et al.  Estimation and Inference in Nonparametric Frontier Models: Recent Developments and Perspectives , 2013 .

[61]  Judit Bar-Ilan,et al.  Citations to the “Introduction to informetrics” indexed by WOS, Scopus and Google Scholar , 2010, Scientometrics.

[62]  Michael Burton,et al.  Core journals: A reappraisal of the Diamond list , 1995 .

[63]  Johan Bollen,et al.  Usage Impact Factor: the effects of sample characteristics on usage-based impact metrics , 2006, J. Assoc. Inf. Sci. Technol..

[64]  Martin Strobel,et al.  The Ranking of Economics Journals by a Tournament Method , 2007 .

[65]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007, J. Assoc. Inf. Sci. Technol..

[66]  Finn R. Førsund,et al.  Farrell revisited–Visualizing properties of DEA production frontiers , 2009, J. Oper. Res. Soc..

[67]  Yolanda K. Kodrzycki,et al.  New Approaches to Ranking Economics Journals , 2006 .

[68]  Léopold Simar,et al.  Non-parametric tests of returns to scale , 2002, Eur. J. Oper. Res..