Auditing citation polarization during the COVID-19 pandemic

The recent pandemic stimulated scientists to publish a significant amount of research that created a surge of citations of COVID-19-related papers in a short time, leading to an abrupt inflation of the journal impact factor (IF). By auditing the complete set of COVID-19-related publications in the Web of Science, we reveal here that COVID-19-related research worsened the polarization of academic journals: the IF before the pandemic was proportional to the increment of IF, which had the effect of increasing inequality while retaining the journal rankings. We also found that the most highly cited studies related to COVID-19 were published in prestigious journals at the onset of the epidemic, independent of their innate importance or quality. Through the present quantitative investigation, our findings caution against the belief that quantitative metrics, particularly IF, can indicate the significance of individual papers. Rather, such metrics reflect the social attention given to a particular study.

[1]  O. Zimba,et al.  Bibliometric and Altmetric Analysis of Retracted Articles on COVID-19 , 2022, Journal of Korean medical science.

[2]  K. Boyack,et al.  Massive covidization of research citations and the citation elite , 2022, medRxiv.

[3]  Woo-Sung Jung,et al.  Disturbance of questionable publishing to academia , 2021, J. Informetrics.

[4]  A. El-Menyar,et al.  Publications and retracted articles of COVID-19 pharmacotherapy-related research: A systematic review , 2021, Science progress.

[5]  Cornelia Lawson,et al.  The funding-productivity-gender nexus in science, a multistage analysis , 2021 .

[6]  A. Sutton,et al.  Following the science? Comparison of methodological and reporting quality of covid-19 and other research from the first wave of the pandemic , 2021, BMC Medicine.

[7]  J. P. Andersen,et al.  Global citation inequality is on the rise , 2021, Proceedings of the National Academy of Sciences.

[8]  Amit N. Patel,et al.  Retraction: Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19. N Engl J Med. DOI: 10.1056/NEJMoa2007621. , 2020, The New England journal of medicine.

[9]  Amit N. Patel,et al.  Retraction—Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis , 2020, The Lancet.

[10]  Albert-László Barabási,et al.  Historical comparison of gender inequality in scientific careers across countries and disciplines , 2019, Proceedings of the National Academy of Sciences.

[11]  A. van de Rijt,et al.  The Matthew effect in science funding , 2018, Proceedings of the National Academy of Sciences.

[12]  Loet Leydesdorff,et al.  Skewness of citation impact data and covariates of citation distributions: A large-scale empirical analysis based on Web of Science data , 2016, J. Informetrics.

[13]  Vincent Larivière,et al.  A simple proposal for the publication of journal citation distributions , 2016, bioRxiv.

[14]  C. Bradshaw,et al.  How to Rank Journals , 2016, PloS one.

[15]  Matjaz Perc,et al.  The Matthew effect in empirical data , 2014, Journal of The Royal Society Interface.

[16]  Soong Moon Kang,et al.  Field experiments of success-breeds-success dynamics , 2014, Proceedings of the National Academy of Sciences.

[17]  Jian Wang,et al.  Unpacking the Matthew effect in citations , 2014, J. Informetrics.

[18]  D. Plenz,et al.  powerlaw: A Python Package for Analysis of Heavy-Tailed Distributions , 2013, PloS one.

[19]  D. Ruths,et al.  Flows of Research Manuscripts Among Scientific Journals Reveal Hidden Submission Patterns , 2012, Science.

[20]  Vincent Larivière,et al.  The weakening relationship between the impact factor and papers' citations in the digital age , 2012, J. Assoc. Inf. Sci. Technol..

[21]  Hendrik P. van Dalen,et al.  Intended and Unintended Consequences of a Publish-or-Perish Culture: A Worldwide Survey , 2012, J. Assoc. Inf. Sci. Technol..

[22]  Ismael Rafols,et al.  How journal rankings can suppress interdisciplinary research: A comparison between Innovation Stud , 2012 .

[23]  Henk F. Moed,et al.  Measuring contextual citation impact of scientific journals , 2009, J. Informetrics.

[24]  Vincent Larivière,et al.  The impact factor's Matthew Effect: A natural experiment in bibliometrics , 2009, J. Assoc. Inf. Sci. Technol..

[25]  Woo-Sung Jung,et al.  Quantitative and empirical demonstration of the Matthew effect in a study of career longevity , 2008, Proceedings of the National Academy of Sciences.

[26]  D. Pendlebury The use and misuse of journal metrics and other citation indicators , 2009, Archivum Immunologiae et Therapiae Experimentalis.

[27]  Thed N. van Leeuwen,et al.  Characteristics of journal impact factors: The effects of uncitedness and citation distribution on the understanding of journal impact factors , 2005, Scientometrics.

[28]  J. S. Long,et al.  Cumulative Advantage and Inequality in Science , 1982 .

[29]  E. Garfield Citation analysis as a tool in journal evaluation. , 1972, Science.

[30]  R. Merton The Matthew effect in science. The reward and communication systems of science are considered. , 1968, Science.

[31]  C. Gini Variabilità e mutabilità : contributo allo studio delle distribuzioni e delle relazioni statistiche , 1912 .