Geometric journal impact factors correcting for individual highly cited articles

Journal impact factors (JIFs) are widely used and promoted but have important limitations. In particular, JIFs can be unduly influenced by individual highly cited articles and hence are inherently unstable. A logical way to reduce the impact of individual high citation counts is to use the geometric mean rather than the standard mean in JIF calculations. Based upon journal rankings 2004-2014 in 50 sub-categories within 5 broad categories, this study shows that journal rankings based on JIF variants tend to be more stable over time if the geometric mean is used rather than the standard mean. The same is true for JIF variants using Mendeley reader counts instead of citation counts. Thus, although the difference is not large, the geometric mean is recommended instead of the arithmetic mean for future JIF calculations. In addition, Mendeley readership-based JIF variants are as stable as those using Scopus citations, confirming the value of Mendeley readership as an academic impact indicator.

[1]  Michael H. MacRoberts,et al.  Problems of citation analysis: A critical review , 1989, JASIS.

[2]  Michelle D. Steward,et al.  A Comprehensive Analysis of Marketing Journal Rankings , 2010 .

[3]  Cassidy R. Sugimoto,et al.  Do Altmetrics Work? Twitter and Ten Other Social Web Services , 2013, PloS one.

[4]  Mike Thelwall,et al.  When are readership counts as useful as citation counts? Scopus versus Mendeley for LIS journals , 2016, J. Assoc. Inf. Sci. Technol..

[5]  Mike Thelwall,et al.  Validating online reference managers for scholarly impact measurement , 2011, Scientometrics.

[6]  Mike Thelwall,et al.  Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics , 2014, J. Assoc. Inf. Sci. Technol..

[7]  T. E. Nisonger,et al.  The Perception of Library and Information Science Journals by LIS Education Deans and ARL Library Directors: A Replication of the Kohl–Davis Study , 2005 .

[8]  Per O. Seglen,et al.  The Skewness of Science , 1992, J. Am. Soc. Inf. Sci..

[9]  Gordana Budimir,et al.  Assessment of research fields in Scopus and Web of Science in the view of national research evaluation in Slovenia , 2013, Scientometrics.

[10]  Alexander Serenko,et al.  Comparing the expert survey and citation impact journal ranking methods: Example from the field of Artificial Intelligence , 2011, J. Informetrics.

[11]  P. Seglen,et al.  Education and debate , 1999, The Ethics of Public Health.

[12]  Vincent Larivière,et al.  The weakening relationship between the impact factor and papers' citations in the digital age , 2012, J. Assoc. Inf. Sci. Technol..

[13]  Carl T. Bergstrom,et al.  The Eigenfactor™ Metrics , 2008, The Journal of Neuroscience.

[14]  Vincent Larivière,et al.  History of the journal impact factor: Contingencies and consequences , 2009, Scientometrics.

[15]  Stefanie Haustein,et al.  Exploring data quality and retrieval strategies for mendeley reader counts , 2015 .

[16]  Joel A. C. Baum,et al.  Free-Riding on Power Laws: questioning the validity of the Impact Factor as a measure of research quality in organization studies , 2011 .

[17]  Mourad Touzani,et al.  Ranking marketing journals using the Google Scholar-based hg-index , 2010, J. Informetrics.

[18]  D. Greenwood,et al.  Selective attrition and bias in a longitudinal health survey among survivors of a disaster , 2007, BMC medical research methodology.

[19]  Pantelis Kalaitzidakis,et al.  An Updated Ranking of Academic Journals in Economics (Une Mise À Jour De L'Ordonnancement Des Revues Scientifiques En Économie)  , 2011 .

[20]  Mike Thelwall.  Journal impact evaluation: a webometric perspective , 2012, Scientometrics.

[21]  Mike Thelwall,et al.  Distributions for cited articles from individual subjects and years , 2014, J. Informetrics.

[22]  Pantelis Kalaitzidakis,et al.  RANKINGS OF ACADEMIC JOURNALS AND INSTITUTIONS IN ECONOMICS , 2003 .

[23]  Jonas Lundberg,et al.  Lifting the crown - citation z-score , 2007, J. Informetrics.

[24]  Euan A. Adie,et al.  Altmetric: enriching scholarly content with article‐level discussion and metrics , 2013, Learn. Publ..

[25]  M. Way,et al.  The San Francisco Declaration on Research Assessment , 2013, Journal of Cell Science.

[26]  Vincent Larivière,et al.  The impact factor's Matthew Effect: A natural experiment in bibliometrics , 2009, J. Assoc. Inf. Sci. Technol..

[27]  R. Cagan.  The San Francisco Declaration on Research Assessment , 2013, Disease Models & Mechanisms.

[28]  Jong Yong Abdiel Foo,et al.  Implications of a Single Highly Cited Article on a Journal and Its Citation Indexes: A Tale of Two Journals , 2013, Accountability in research.

[29]  Vincent Larivière,et al.  Who reads research articles? An altmetrics analysis of Mendeley user categories , 2015, J. Assoc. Inf. Sci. Technol..

[30]  Hannah Brown,et al.  How impact factors changed medical publishing—and science , 2007, BMJ : British Medical Journal.

[31]  N. Adler,et al.  When Knowledge Wins: Transcending the Sense and Nonsense of Academic Rankings , 2009 .

[32]  M. Sales-Pardo,et al.  Effectiveness of Journal Ranking Schemes as a Tool for Locating Information , 2008, PloS one.

[33]  Massimo Franceschet,et al.  The first Italian research assessment exercise: A bibliometric perspective , 2009, J. Informetrics.

[34]  Per Ottar Seglen,et al.  Causal relationship between article citedness and journal impact , 1994 .

[35]  Tobias Siebenlist,et al.  Applying social bookmarking data to evaluate journal usage , 2011, J. Informetrics.

[36]  Jerome K. Vanclay,et al.  Impact factor: outdated artefact or stepping-stone to journal certification? , 2011, Scientometrics.

[37]  Qais Al-Awqati.  Impact factors and prestige. , 2007, Kidney international.

[38]  Mark D. Miller,et al.  Examining differences across journal rankings , 2005, CACM.

[39]  Mike Thelwall,et al.  Can Mendeley bookmarks reflect readership? A survey of user motivations , 2016, J. Assoc. Inf. Sci. Technol..

[40]  Paul Genoni,et al.  Citation analysis and peer ranking of Australian social science journals , 2010, Scientometrics.

[41]  Rodrigo Costas,et al.  How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications , 2014, Scientometrics.

[42]  Jeremy B. Fox,et al.  A multi-method evaluation of journals in the decision and management sciences by US academics , 2000 .

[43]  Peter T. Shepherd.  The feasibility of developing and implementing journal usage factors: a research project sponsored by UKSG , 2007 .

[44]  Michel Zitt,et al.  The journal impact factor: angel, devil, or scapegoat? A comment on J.K. Vanclay’s article 2011 , 2012, Scientometrics.

[45]  Evaristo Jiménez-Contreras,et al.  Ranking of departments and researchers within a university using two different databases: Web of Science versus Scopus , 2009, Scientometrics.

[46]  M. Robinson,et al.  Small-sample estimation of negative binomial dispersion, with applications to SAGE data. , 2007, Biostatistics.

[47]  Mike Thelwall,et al.  Mendeley readership altmetrics for medical articles: An analysis of 45 fields , 2016, J. Assoc. Inf. Sci. Technol..