Mitigating ageing bias in article level metrics using citation network analysis

Abstract Article level scientometric indicators (ALMs) are usually of cumulative nature making articles of different age hard to compare. Here, we introduce a new ALM, the Time Debiased Significance Score (TDSS), which measures the significance of a publication based on the structure of the whole citation network and eliminates the global ageing bias in the network: older publications should not be a priori privileged or disadvantaged compared to newer ones. The TDSS is based on a modified variant of the PageRank measure, incorporating a mathematically consistent temporal detrending and ensuring a few key features: (i) the TDSS should not show any global trend as a function of the topological index (causal order in the citation network); (ii) the TDSS value of a publication should decrease as time passes (and the citation network grows) if no more citations are associated with it. The above definition is beneficial in multiple ways, including e.g. low computational complexity and weak domain dependence. Further, estimation of reliability of the TDSS and its extension to groups of items like overall score of a research group are also possible.

[1]  E. Garfield Fortnightly Review: How can impact factors be improved? , 1996 .

[2]  Joel Nothman,et al.  SciPy 1.0-Fundamental Algorithms for Scientific Computing in Python , 2019, ArXiv.

[3]  Ludo Waltman,et al.  Field normalization of scientometric indicators , 2018, Springer Handbook of Science and Technology Indicators.

[4]  Philip M. Davis Eigenfactor: Does the principle of repeated improvement result in better estimates than raw citation counts? , 2008 .

[5]  Anthony F. J. van Raan,et al.  Universality of citation distributions revisited , 2011, J. Assoc. Inf. Sci. Technol..

[6]  John D. Hunter,et al.  Matplotlib: A 2D Graphics Environment , 2007, Computing in Science & Engineering.

[7]  L. Egghe,et al.  Theory and practise of the g-index , 2006, Scientometrics.

[8]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[9]  Michel Zitt,et al.  Modifying the journal impact factor by fractional citation weighting: The audience factor , 2008, J. Assoc. Inf. Sci. Technol..

[10]  Sergei Maslov,et al.  Ranking scientific publications using a model of network traffic , 2006, ArXiv.

[11]  Zoltán Néda,et al.  Science and Facebook: The same popularity law! , 2017, PloS one.

[12]  S. Nadarajah,et al.  Extreme Value Distributions: Theory and Applications , 2000 .

[13]  Tibor Braun,et al.  Cross-field normalization of scientometric indicators , 1996, Scientometrics.

[14]  J. Hirsch Does the h index have predictive power? , 2007, Proceedings of the National Academy of Sciences.

[15]  Carl T. Bergstrom,et al.  The Science of Science , 2018, Science.

[16]  Travis E. Oliphant,et al.  Python for Scientific Computing , 2007, Computing in Science & Engineering.

[17]  P. Seglen,et al.  Education and debate , 1999, The Ethics of Public Health.

[18]  Ludo Waltman,et al.  Source normalized indicators of citation impact: an overview of different approaches and an empirical comparison , 2012, Scientometrics.

[19]  Qing Ke,et al.  Defining and identifying Sleeping Beauties in science , 2015, Proceedings of the National Academy of Sciences.

[20]  Michael Golosovsky,et al.  Runaway events dominate the heavy tail of citation distributions , 2012, ArXiv.

[21]  Eugene Garfield,et al.  The Impact Factor and Using It Correctly , 2002 .

[22]  Dalibor Fiala Current index: A Proposal for a dynamic rating system for researchers , 2014, J. Assoc. Inf. Sci. Technol..

[23]  Ludo Waltman,et al.  A review of the literature on citation impact indicators , 2015, J. Informetrics.

[24]  Krešimir Šolić,et al.  An Information Security and Privacy Self-Assessment (ISPSA) Tool for Internet Users , 2015 .

[25]  Ludo Waltman,et al.  PageRank-Related Methods for Analyzing Citation Networks , 2014 .

[26]  Johan Bollen,et al.  Journal status , 2006, Scientometrics.

[27]  Gaël Varoquaux,et al.  The NumPy Array: A Structure for Efficient Numerical Computation , 2011, Computing in Science & Engineering.

[28]  E. Garfield The history and meaning of the journal impact factor. , 2006, JAMA.

[29]  J. Wawer How to stop salami science: promotion of healthy trends in publishing behavior , 2018, Accountability in research.

[30]  Konrad Paul Kording,et al.  Future impact: Predicting scientific success , 2012, Nature.

[31]  T. Opthof,et al.  Sense and nonsense about the impact factor. , 1997, Cardiovascular research.

[32]  Ludo Waltman,et al.  Predicting the long-term citation impact of recent publications , 2015, J. Informetrics.

[33]  R. Rousseau,et al.  The R- and AR-indices: Complementing the h-index , 2007 .

[34]  Santo Fortunato,et al.  Characterizing and Modeling Citation Dynamics , 2011, PloS one.

[35]  Jorge J. Moré,et al.  The Levenberg-Marquardt algo-rithm: Implementation and theory , 1977 .

[36]  Philip M. Davis Eigenfactor : Does the Principle of Repeated Improvement Result in Better Journal Impact Estimates than Raw Citation Counts? , 2008, J. Assoc. Inf. Sci. Technol..

[37]  Santo Fortunato,et al.  On the Predictability of Future Impact in Science , 2013, Scientific Reports.

[38]  Olle Persson,et al.  The DCI index: Discounted cumulated impact-based research evaluation , 2008, J. Assoc. Inf. Sci. Technol..

[39]  ANTHONY F. J. VAN RAAN,et al.  Sleeping Beauties in science , 2004, Scientometrics.

[40]  Péter Pollner,et al.  Scientometrics: Untangling the topics , 2014, ArXiv.

[41]  Dalibor Fiala,et al.  Ageing of Edges in Collaboration Networks and its Effect on Author Rankings , 2015 .

[42]  Gabriel Pinski,et al.  Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics , 1976, Inf. Process. Manag..

[43]  Jean-Loup Guillaume,et al.  Fast unfolding of communities in large networks , 2008, 0803.0476.

[44]  Santo Fortunato,et al.  Attention Decay in Science , 2015, J. Informetrics.

[45]  Rodrigo Costas,et al.  Do “altmetrics” correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective , 2014, J. Assoc. Inf. Sci. Technol..

[46]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[47]  D. Pendlebury The use and misuse of journal metrics and other citation indicators , 2009, Archivum Immunologiae et Therapiae Experimentalis.

[48]  Lise Getoor,et al.  FutureRank: Ranking Scientific Articles by Predicting their Future PageRank , 2009, SDM.

[49]  Michael Golosovsky,et al.  Stochastic dynamical model of a growing network based on self-exciting point process , 2012, Physical review letters.

[50]  E Garfield,et al.  [The impact factor and its proper application]. , 1998, Der Unfallchirurg.

[51]  Claudio Castellano,et al.  Universality of citation distributions: Toward an objective measure of scientific impact , 2008, Proceedings of the National Academy of Sciences.

[52]  Carl T. Bergstrom,et al.  The Eigenfactor™ Metrics , 2008, The Journal of Neuroscience.

[53]  Iman Tahamtan,et al.  Factors affecting number of citations: a comprehensive review of the literature , 2016, Scientometrics.

[54]  Ronald N. Kostoff Citation analysis cross-field normalization: A new paradigm , 2006, Scientometrics.

[55]  Albert-László Barabási,et al.  Quantifying Long-Term Scientific Impact , 2013, Science.