Multiple Citation Indicators and Their Composite across Scientific Disciplines

Many fields face an increasing prevalence of multi-authorship, and this poses challenges in assessing citation metrics. Here, we explore multiple citation indicators that address total impact (number of citations, Hirsch H index [H]), co-authorship adjustment (Schreiber Hm index [Hm]), and author order (total citations to papers as single; single or first; or single, first, or last author). We demonstrate the correlation patterns between these indicators across 84,116 scientists (those among the top 30,000 for impact in a single year [2013] in at least one of these indicators) and separately across 12 scientific fields. Correlation patterns vary across these 12 fields. In physics, total citations are highly negatively correlated with indicators of co-authorship adjustment and of author order, while in other sciences the negative correlation is seen only for total citation impact and citations to papers as single author. We propose a composite score that sums standardized values of these six log-transformed indicators. Of the 1,000 top-ranked scientists with the composite score, only 322 are in the top 1,000 based on total citations. Many Nobel laureates and other extremely influential scientists rank among the top-1,000 with the composite indicator, but would rank much lower based on total citations. Conversely, many of the top 1,000 authors on total citations have had no single/first/last-authored cited paper. More Nobel laureates of 2011–2015 are among the top authors when authors are ranked by the composite score than by total citations, H index, or Hm index; 40/47 of these laureates are among the top 30,000 by at least one of the six indicators. We also explore the sensitivity of indicators to self-citation and alphabetic ordering of authors in papers across different scientific fields. Multiple indicators and their composite may give a more comprehensive picture of impact, although no citation indicator, single or composite, can be expected to select all the best scientists.

[1]  Marek Kosmulski,et al.  The order in the lists of authors in multi-author papers revisited , 2012, J. Informetrics.

[2]  Henk F. Moed,et al.  Bibliometric Indicators Reflect Publication and Management Strategies , 2000, Scientometrics.

[3]  Filippo Radicchi,et al.  The Possible Role of Resource Requirements and Academic Career-Choice Risk on Gender Differences in Publication Rate and Impact , 2012, PloS one.

[4]  Tove Faber Frandsen,et al.  What is in a name? Credit assignment practices in different disciplines , 2010, J. Informetrics.

[5]  Kevin W Boyack,et al.  A list of highly influential biomedical researchers, 1996–2011 , 2013, European journal of clinical investigation.

[6]  Wei Huang,et al.  DO ABCs GET MORE CITATIONS THAN XYZs , 2015 .

[7]  Luís A Nunes Amaral,et al.  Cross-evaluation of metrics to estimate the significance of creative works , 2015, Proceedings of the National Academy of Sciences.

[8]  M. Sales-Pardo,et al.  Effectiveness of Journal Ranking Schemes as a Tool for Locating Information , 2008, PloS one.

[9]  Marta Sales-Pardo,et al.  Statistical validation of a global model for the distribution of the ultimate number of citations accrued by papers published in a scientific journal , 2010 .

[10]  Benjamin F. Jones,et al.  Supporting Online Material Materials and Methods Figs. S1 to S3 References the Increasing Dominance of Teams in Production of Knowledge , 2022 .

[11]  Erica Frank,et al.  Significance of Authorship Position: An Open-Ended International Assessment , 2011, The American journal of the medical sciences.

[12]  Luis A. Nunes Amaral,et al.  The Distribution of the Asymptotic Number of Citations to Sets of Publications by a Researcher or from an Academic Department Are Consistent with a Discrete Lognormal Model , 2015, PloS one.

[13]  Santo Fortunato,et al.  On the Predictability of Future Impact in Science , 2013, Scientific Reports.

[14]  Leo Egghe,et al.  Mathematical theory of the h- and g-index in case of fractional counting of authorship , 2008, J. Assoc. Inf. Sci. Technol..

[15]  J. Hirsch.  Does the h index have predictive power? , 2007, Proceedings of the National Academy of Sciences.

[16]  Loet Leydesdorff.  How are new citation-based journal indicators adding to the bibliometric toolbox? , 2009 .

[17]  John P. A. Ioannidis,et al.  Assessing value in biomedical research: the PQRST of appraisal and reward. , 2014, JAMA.

[18]  L. Egghe,et al.  Theory and practise of the g-index , 2006, Scientometrics.

[19]  Amin Mazloumian,et al.  Predicting Scholars' Scientific Impact , 2012, PloS one.

[20]  Hirotaka Kawashima,et al.  Accuracy evaluation of Scopus Author ID based on the largest funding database in Japan , 2015, Scientometrics.

[21]  R. Slone.  Coauthors' contributions to major papers published in the AJR: frequency of undeserved coauthorship. , 1996, AJR. American journal of roentgenology.

[22]  L. Bornmann,et al.  The state of h index research , 2009, EMBO reports.

[23]  Peter van den Besselaar,et al.  Author disambiguation using multi-aspect similarity indicators , 2011, Scientometrics.

[24]  Mônica G. Campiteli,et al.  Is it possible to compare researchers with different scientific interests? , 2006, Scientometrics.

[25]  S. Rijcke,et al.  Bibliometrics: The Leiden Manifesto for research metrics , 2015, Nature.

[26]  Lorna Elizabeth Wildgaard,et al.  A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar , 2015, Scientometrics.

[27]  John P. A. Ioannidis,et al.  Measuring Co-Authorship and Networking-Adjusted Scientific Impact , 2008, PloS one.

[28]  A. D. Jackson,et al.  Measures for measures , 2006, Nature.

[29]  Elise Smith,et al.  Authorship and Responsibility in Health Sciences Research: A Review of Procedures for Fairly Allocating Authorship in Multi-Author Studies , 2011, Science and Engineering Ethics.

[30]  Richard S. J. Tol,et al.  Credit where credit’s due: accounting for co-authorship in citation counts , 2011, Scientometrics.

[31]  M. Jennions,et al.  The h index and career assessment by numbers. , 2006, Trends in ecology & evolution.

[32]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[33]  Marcia McNutt,et al.  The measure of research merit , 2014, Science.

[34]  Mônica G. Campiteli,et al.  An index to quantify an individual's scientific research valid across disciplines , 2005 .

[35]  Tang Xiaoli,et al.  Perceptions of author order versus contribution among researchers with different professional ranks and the potential of harmonic counts for encouraging ethical co-authorship practices , 2013, Scientometrics.

[36]  Kevin W. Boyack,et al.  Including cited non-source items in a large-scale map of science: What difference does it make? , 2014, J. Informetrics.

[37]  Robert Aboukhalil,et al.  The rising trend in authorship , 2014 .

[38]  Rodrigo Costas,et al.  Do age and professional rank influence the order of authorship in scientific publications? Some evidence from a micro-level perspective , 2011, Scientometrics.

[39]  Claudio Castellano,et al.  Analysis of bibliometric indicators for individual scholars in a large data set , 2013, Scientometrics.

[40]  John P. A. Ioannidis,et al.  How to Make More Published Research True , 2014, PLoS medicine.

[41]  J. Ioannidis,et al.  Estimates of the Continuously Publishing Core in the Scientific Workforce , 2014, PloS one.

[42]  Ludo Waltman,et al.  An empirical analysis of the use of alphabetical authorship in scientific publishing , 2012, J. Informetrics.

[43]  V. Larivière,et al.  Design and Update of a Classification System: The UCSD Map of Science , 2012, PloS one.

[44]  Michael Schreiber,et al.  A modification of the h-index: The hm-index accounts for multi-authored manuscripts , 2008, J. Informetrics.