Who Is the 'Journal Grand Master'? A New Ranking Based on the Elo Rating System

In this paper we transfer the Elo rating system, which is widely accepted in chess, sports and other disciplines, to rank scientific journals. The advantage of the Elo system is the explicit consideration of the factor time or the history of a journal’s performance. Most other rankings that are commonly applied neglect this fact. The Elo ranking approach can easily be applied to any metric, published on a regular basis, to rank journals. We illustrate the approach using the SNIP indicator based on citation data from Scopus. Our balanced panel consists of 7,748 journals from many scientific fields for the period from 1999 to 2015. We show that the Elo approach produces a similar but not identical ranking compared to other rankings based on the SNIP. Especially the rank order for rather ’middle-class’ journals can tremendously change.

[1]  Thed N. van Leeuwen,et al.  Some modifications to the SNIP journal impact indicator , 2012, J. Informetrics.

[2]  Wolfgang Glänzel,et al.  Statistical reliability of comparisons based on the citation impact of scientific publications , 2005, Scientometrics.

[3]  Margaret Wu,et al.  Estimating the accuracies of journal impact factor through bootstrap , 2014, J. Informetrics.

[4]  D. Greenwood,et al.  Selective attrition and bias in a longitudinal health survey among survivors of a disaster , 2007, BMC medical research methodology.

[5]  Mike Thelwall,et al.  The accuracy of confidence intervals for field normalised indicators , 2017, J. Informetrics.

[6]  Jerome K. Vanclay,et al.  Impact factor: outdated artefact or stepping-stone to journal certification? , 2011, Scientometrics.

[7]  A. Elo.  The rating of chessplayers, past and present , 1978 .

[8]  Loet Leydesdorff,et al.  Scopus' SNIP indicator: Reply to Moed , 2011, J. Assoc. Inf. Sci. Technol..

[9]  Hal S. Stern,et al.  Designing a College Football Playoff System , 1999 .

[10]  David I. Stern,et al.  Uncertainty Measures for Economics Journal Impact Factors , 2012 .

[11]  Henk F. Moed,et al.  Citation-based metrics are appropriate tools in journal assessment provided that they are accurate and used in an informed way , 2012, Scientometrics.

[12]  Ronald Rousseau,et al.  A quick and easy method to estimate the random effect on citation measures , 2005, Scientometrics.

[13]  T. Opthof,et al.  Sense and nonsense about the impact factor. , 1997, Cardiovascular research.

[14]  Jean-François Laslier,et al.  Tournament Solutions And Majority Voting , 1997 .

[15]  Henk F. Moed,et al.  Measuring contextual citation impact of scientific journals , 2009, J. Informetrics.

[16]  Marjan Mernik,et al.  A comparison between different chess rating systems for ranking evolutionary algorithms , 2014, 2014 Federated Conference on Computer Science and Information Systems.

[17]  László Á. Kóczy,et al.  The World Cup of Economics Journals: A Ranking by a Tournament Method , 2010 .

[18]  Rating systems based on paired comparison models , 1991 .