Corrective factors for author- and journal-based metrics impacted by citations to accommodate for retractions

Citation-based metrics are frequently used to evaluate the level, or quality, of a researcher, or their work, often as a function of the ranking of the journal in which they publish, and broadly tend to be divided into journal-based metrics (JBMs) and author-based metrics (ABMs). Despite wide knowledge of the gaming of such metrics, in particular the Clarivate Analytics journal impact factor (JIF), no suitable substitute concept has yet emerged, nor has any corrective measure been developed. In a post-publication peer review world of increasing retractions, and within a framework of open science, we propose correction factors for JBMs and ABMs that take into account retractions. We describe ways to correct the JIF, CiteScore, the 5-year Impact Factor, Immediacy Index, Cited Half-Life, Raw Impact per Paper and other JBMs (Eigenfactor Score and Article Influence Score) as well as the h-index, one of the most widespread ABMs, depending on the number of retractions for that journal or individual, respectively. The existence of such corrective factors could make the use of these metrics more transparent, and might allow them to be used in a world that is adapting to an increase in retractions and corrective measures to deal with erroneous scientific literature. We caution that such correction factors should be used exclusively as such, and should not be viewed, or used, as punitive factors.

[1]  J. A. Teixeira da Silva,et al.  Compounding Error: The Afterlife of Bad Science , 2017 .

[2]  C. Blanford.  Impact factors, citation distributions and journal stratification , 2016, Journal of Materials Science.

[3]  L. Bornmann,et al.  The state of h index research , 2009, EMBO reports.

[4]  J. A. Teixeira da Silva,et al.  Ending the retraction stigma: Encouraging the reporting of errors in the biomedical record , 2019, Research Ethics.

[5]  William H. Walters,et al.  Do subjective journal ratings represent whole journals or typical articles? Unweighted or weighted citation impact? , 2017, J. Informetrics.

[6]  Arturo Casadevall,et al.  Why Has the Number of Scientific Retractions Increased? , 2013, PloS one.

[7]  Jaime A. Teixeira da Silva,et al.  Why do some retracted papers continue to be cited? , 2016, Scientometrics.

[8]  Ludo Waltman,et al.  A review of the literature on citation impact indicators , 2015, J. Informetrics.

[9]  J. Brooks.  Why most published research findings are false: Ioannidis JP, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece , 2008 .

[10]  Jaime A. Teixeira da Silva,et al.  Notices and Policies for Retractions, Expressions of Concern, Errata and Corrigenda: Their Importance, Content, and Context , 2016, Science and Engineering Ethics.

[11]  C. Wiedermann.  Inaction over retractions of identified fraudulent publications: ongoing weakness in the system of scientific self-correction , 2018, Accountability in research.

[12]  D. Baulcombe,et al.  An enhanced transient expression system in plants based on suppression of gene silencing by the p19 protein of tomato bushy stunt virus. , 2003, The Plant journal : for cell and molecular biology.

[13]  Jaime A. Teixeira da Silva.  The Google Scholar h-index: useful but burdensome metric , 2018, Scientometrics.

[14]  Elizabeth Wager,et al.  Retraction policies of top scientific journals ranked by impact factor. , 2015, Journal of the Medical Library Association : JMLA.

[15]  G Winkmann,et al.  Medizinisch-biowissenschaftliche Datenbanken und der Impact-Faktor , 2000 .

[16]  D. Fanelli.  Why Growing Retractions Are (Mostly) a Good Sign , 2013, PLoS medicine.

[17]  Mohammad Hosseini,et al.  Doing the Right Thing: A Qualitative Investigation of Retractions Due to Unintentional Error , 2017, Science and Engineering Ethics.

[18]  Minghua Zhang,et al.  A Comprehensive Survey of Retracted Articles from the Scholarly Literature , 2012, PloS one.

[19]  Manolis Antonoyiannakis,et al.  How a Single Paper Affects the Impact Factor: Implications for Scholarly Publishing , 2019, ISSI.

[20]  Jaime A. Teixeira da Silva,et al.  Multiple versions of the h-index: cautionary use for formal academic purposes , 2018, Scientometrics.

[21]  Judit Bar-Ilan,et al.  Temporal characteristics of retracted articles , 2018, Scientometrics.

[22]  Jaime A. Teixeira da Silva,et al.  Citation inflation: the effect of not correcting the scientific literature sufficiently, a case study in the plant sciences , 2018, Scientometrics.

[23]  D. Resnik,et al.  An International Study of Research Misconduct Policies , 2015, Accountability in research.

[24]  J. A. T. Silva,et al.  Academics must list all publications on their CV , 2018 .

[25]  M. Matsuda,et al.  Visfatin: a protein secreted by visceral fat that mimics the effects of insulin. , 2005, Science.

[27]  Javier Franco Aixelá,et al.  Publishing and impact criteria, and their bearing on Translation Studies: In search of comparability , 2015 .

[28]  Jaime A. Teixeira da Silva,et al.  Is the soundness-only quality control policy of open access mega journals linked to a higher rate of published errors? , 2019, Scientometrics.

[29]  Jaime A. Teixeira da Silva,et al.  CiteScore: A cite for sore eyes, or a valuable, transparent metric? , 2017, Scientometrics.

[30]  F. Paulus,et al.  The Impact Factor Fallacy , 2017, bioRxiv.

[31]  Judit Bar-Ilan,et al.  Post retraction citations in context: a case study , 2017, Scientometrics.

[32]  Xue-Li Liu,et al.  Journal Impact Factor: Do the Numerator and Denominator Need Correction? , 2016, PloS one.

[33]  Kihong Kim,et al.  Overview of journal metrics , 2018 .

[34]  E. Garfield.  Citation analysis as a tool in journal evaluation. , 1972, Science.

[35]  J. A. Teixeira da Silva.  The Google Scholar h-index: useful but burdensome metric , 2018, Scientometrics.

[36]  François-Xavier Coudert,et al.  Correcting the Scientific Record: Retraction Practices in Chemistry and Materials Science , 2019, Chemistry of Materials.

[37]  Björn Hammarfelt,et al.  Indicators as judgment devices : An empirical study of citizen bibliometrics in research evaluation , 2017 .

[38]  Jaime A. Teixeira da Silva,et al.  Rejoinder to “Multiple versions of the h-index: cautionary use for formal academic purposes” , 2018, Scientometrics.

[39]  Rodrigo Costas,et al.  The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level , 2007, J. Informetrics.

[40]  Xingshun Qi,et al.  Characteristics of retractions related to faked peer reviews: an overview , 2016, Postgraduate Medical Journal.

[41]  Francisco Herrera,et al.  h-Index: A review focused in its variants, computation and standardization for different scientific fields , 2009, J. Informetrics.

[42]  Ernesto Roldan-Valadez,et al.  Current concepts on bibliometrics: a brief review about impact factor, Eigenfactor score, CiteScore, SCImago Journal Rank, Source-Normalised Impact per Paper, H-index, and alternative metrics , 2018, Irish Journal of Medical Science (1971 -).

[43]  Akira Ukawa,et al.  Repeating probability of authors with retracted scientific publications , 2018, Accountability in research.

[44]  Jaime A. Teixeira da Silva,et al.  Clarivate Analytics: Continued Omnia vanitas Impact Factor Culture , 2018, Sci. Eng. Ethics.

[45]  David Colquhoun,et al.  An investigation of the false discovery rate and the misinterpretation of p-values , 2014, Royal Society Open Science.

[46]  P. Li,et al.  Role of stromal cells-mediated Notch-1 in the invasion of T-ALL cells. , 2015, Experimental cell research.

[47]  Jaime A. Teixeira da Silva,et al.  Highly cited retracted papers , 2017, Scientometrics.

[48]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[49]  George M. Santangelo,et al.  Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level , 2015, bioRxiv.

[50]  Jaime,et al.  The Importance of Retractions and the Need to Correct the Downstream Literature , 2015 .