Should highly cited items be excluded in impact factor calculation? The effect of review articles on journal impact factor

The use of the journal impact factor for evaluation purposes has been widely challenged because of the simplistic algorithm involved in impact factor calculation. As a result, a journal's impact factor may be affected by many variables, such as highly cited review articles. In the present study, we aimed to investigate whether and to what extent review articles affect the impact factors of journals. We examined the citations of original research articles and review articles in the field of physics, and compared the impact factors of the journals Science , Nature , and Cell with and without citation counts of review articles included. The results showed that review articles consistently received higher numbers of citations than did original research articles. In addition, review articles accounted for 3–10% of the impact factors of these top journals. Based on our findings, we suggest that sophisticated methods, in addition to the impact factor, be employed for research evaluation purposes.

[1]  Jiang Li,et al.  A journal's impact factor is influenced by changes in publication delays of citing journals , 2017, J. Assoc. Inf. Sci. Technol..

[2]  E. Garfield.  The history and meaning of the journal impact factor. , 2006, JAMA.

[3]  M. Roizen,et al.  Hallmarks of Cancer: The Next Generation , 2012 .

[4]  Ben R. Martin,et al.  Editors’ JIF-boosting stratagems – Which are appropriate and which not? , 2016 .

[5]  Jagadeesh Bayry,et al.  Metrics: journal's impact factor skewed by a single paper , 2010, Nature.

[6]  S. Rijcke,et al.  Bibliometrics: The Leiden Manifesto for research metrics , 2015, Nature.

[7]  F. Paulus,et al.  The Impact Factor Fallacy , 2017, bioRxiv.

[8]  E. Garfield.  Citation analysis as a tool in journal evaluation. , 1972, Science.

[9]  Matthew E. Falagas,et al.  The top-ten in journal impact factor manipulation , 2008, Archivum Immunologiae et Therapiae Experimentalis.

[10]  Lei Lei.  When science meets cluttered writing: adjectives and adverbs in academia revisited , 2016, Scientometrics.

[11]  B. A. Boom,et al.  GW170814: A Three-Detector Observation of Gravitational Waves from a Binary Black Hole Coalescence. , 2017, Physical review letters.

[12]  Weishu Liu,et al.  The probability of publishing in first-quartile journals , 2015, Scientometrics.

[13]  Do review articles boost journal impact factors? A longitudinal analysis for five pharmacology journals , 2018, Naunyn-Schmiedeberg's Archives of Pharmacology.

[14]  Chao Zuo,et al.  The effect of publishing a highly cited paper on a journal's impact factor: A case study of the Review of Particle Physics , 2017, Learn. Publ..

[15]  S Fiorucci,et al.  Results from a Search for Dark Matter in the Complete LUX Exposure. , 2016, Physical review letters.

[16]  Erwin Krauskopf.  Deceiving the research community through manipulation of the impact factor , 2013, J. Assoc. Inf. Sci. Technol..

[17]  B. A. Boom,et al.  GW170104: Observation of a 50-Solar-Mass Binary Black Hole Coalescence at Redshift 0.2. , 2017, Physical review letters.

[18]  Lutz Bornmann,et al.  Critical rationalism and the search for standard (field-normalized) indicators in bibliometrics , 2018, J. Informetrics.

[19]  The Ligo Scientific Collaboration,et al.  GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral , 2017, 1710.05832.

[20]  Guang Yu,et al.  An automatic recognition method of journal impact factor manipulation , 2011, J. Inf. Sci..

[21]  Dong-Hui Yang,et al.  Detecting impact factor manipulation with data mining techniques , 2016, Scientometrics.

[22]  Ludo Waltman,et al.  A review of the literature on citation impact indicators , 2015, J. Informetrics.

[23]  Guang Yu,et al.  Reliability-based citation impact factor and the manipulation of impact factor , 2010, Scientometrics.

[24]  Haiyan Wang,et al.  Is it Suitable for a Journal to Bid for Publishing a Review That is Likely to be Highly Cited? , 2019, Sci. Eng. Ethics.

[25]  Lei Lei,et al.  Readability and citations in information science: evidence from abstracts and articles of four journals (2003–2012) , 2016, Scientometrics.

[26]  Jian Wang,et al.  Reviewers are blinkered by bibliometrics , 2017, Nature.

[27]  G. Sheldrick.  A short history of SHELX. , 2008, Acta crystallographica. Section A, Foundations of crystallography.

[28]  Jong Yong Abdiel Foo,et al.  Implications of a Single Highly Cited Article on a Journal and Its Citation Indexes: A Tale of Two Journals , 2013, Accountability in research.

[29]  Haseeb F. Rashid,et al.  Bibliometric Analysis as a Tool in Journal Evaluation , 1991 .