Predicting publication long-term impact through a combination of early citations and journal impact factor

Abstract The ability to predict the long-term impact of a scientific article soon after its publication is of great value towards accurate assessment of research performance. In this work we test the hypothesis that good predictions of long-term citation counts can be obtained through a combination of a publication's early citations and the impact factor of the hosting journal. The test is performed on a corpus of 123,128 WoS publications authored by Italian scientists, using linear regression models. The average accuracy of the prediction is good for citation time windows above two years, decreases for lowly-cited publications, and varies across disciplines. As expected, the role of the impact factor in the combination becomes negligible after only two years from publication.

[1]  Giovanni Abramo,et al.  Refrain from adopting the combination of citation and journal metrics to grade publications, as used in the Italian national research assessment exercise (VQR 2011–2014) , 2016, Scientometrics.

[2]  Giovanni Abramo,et al.  Citations versus journal impact factor as proxy of quality: could the latter ever be preferable? , 2010, Scientometrics.

[3]  Cassidy R. Sugimoto,et al.  Do Altmetrics Work? Twitter and Ten Other Social Web Services , 2013, PloS one.

[4]  Lutz Bornmann,et al.  What do citation counts measure? A review of studies on citing behavior , 2008, J. Documentation.

[5]  Jian Wang,et al.  Citation time window choice for research impact evaluation , 2013, Scientometrics.

[6]  Stephan B. Bruns,et al.  Research assessment using early citation information , 2016, Scientometrics.

[7]  Mike Thelwall,et al.  Evaluating altmetrics , 2013, Scientometrics.

[8]  A. Zeileis.  Econometric Computing with HC and HAC Covariance Matrix Estimators , 2004 .

[9]  Giovanni Abramo,et al.  Revisiting the scientometric conceptualization of impact and its measurement , 2018, J. Informetrics.

[10]  Ronald Rousseau,et al.  Citation distribution of pure mathematics journals , 1988 .

[11]  Wolfgang Glänzel,et al.  Better late than never? On the chance to become highly cited only beyond the standard bibliometric time horizon , 2004, Scientometrics.

[12]  Jonathan Adams,et al.  Early citation counts correlate with accumulated impact , 2005, Scientometrics.

[13]  David I. Stern,et al.  High-Ranked Social Science Journal Articles Can Be Identified from Early Citation Information , 2014, PloS one.

[14]  Ludo Waltman,et al.  Predicting the long-term citation impact of recent publications , 2015, J. Informetrics.

[15]  E. Garfield.  Citation analysis as a tool in journal evaluation. , 1972, Science.

[16]  Albert-László Barabási,et al.  Quantifying Long-Term Scientific Impact , 2013, Science.

[17]  Mike Thelwall,et al.  Mendeley readership counts: An investigation of temporal and disciplinary differences , 2016, J. Assoc. Inf. Sci. Technol..

[18]  Mike Thelwall,et al.  Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics , 2014, J. Assoc. Inf. Sci. Technol..

[19]  M. Sales-Pardo,et al.  Effectiveness of Journal Ranking Schemes as a Tool for Locating Information , 2008, PloS one.

[20]  Yajun Mei,et al.  Comment on “Quantifying long-term scientific impact” , 2014, Science.

[21]  John Mingers,et al.  Exploring the dynamics of journal citations: Modelling with s-curves , 2008, J. Oper. Res. Soc..

[22]  Jian Wang,et al.  How to improve the prediction based on citation impact percentiles for years shortly after the publication date? , 2013, J. Informetrics.

[23]  Tindaro Cicero,et al.  Assessing the varying level of impact measurement accuracy as a function of the citation window length , 2011, J. Informetrics.

[24]  Tindaro Cicero,et al.  Revisiting the scaling of citations for research assessment , 2012, J. Informetrics.

[25]  Mike Thelwall,et al.  Validating online reference managers for scholarly impact measurement , 2011, Scientometrics.

[26]  Loet Leydesdorff,et al.  Group‐based trajectory modeling (GBTM) of citations in scholarly literature: Dynamic qualities of “transient” and “sticky knowledge claims” , 2013, J. Assoc. Inf. Sci. Technol..

[27]  Mike Thelwall,et al.  A combined bibliometric indicator to predict article impact , 2011, Inf. Process. Manag..