On the calculation of percentile-based bibliometric indicators

A percentile-based bibliometric indicator is an indicator that values publications based on their position within the citation distribution of their field. The most straightforward percentile-based indicator is the proportion of frequently cited publications, for instance, the proportion of publications that belong to the top 10% most frequently cited of their field. Recently, more complex percentile-based indicators have been proposed. A difficulty in the calculation of percentile-based indicators is caused by the discrete nature of citation distributions combined with the presence of many publications with the same number of citations. We introduce an approach to calculating percentile-based indicators that deals with this difficulty in a more satisfactory way than earlier approaches suggested in the literature. We show in a formal mathematical framework that our approach leads to indicators that do not suffer from biases in favor of or against particular fields of science.

[1]  Loet Leydesdorff,et al.  Integrated Impact Indicators (I3) compared with Impact Factors (IFs): An alternative research design with policy implications , 2011, J. Assoc. Inf. Sci. Technol..

[2]  Amy M. Hightower,et al.  Science and Engineering Indicators , 1993 .

[3]  Lutz Bornmann,et al.  Further steps towards an ideal method of measuring citation performance: The avoidance of citation (ratio) averages in field-normalization , 2011, J. Informetrics.

[4]  Thed N. van Leeuwen,et al.  The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence , 2003, Scientometrics.

[5]  Dear Sir 1 Percentile Ranks and the Integrated Impact Indicator ( I 3 ) , .

[6]  Thed N. van Leeuwen,et al.  Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference? , 2002, Scientometrics.

[7]  Ronald Rousseau,et al.  Basic properties of both percentile rank scores and the I3 indicator , 2012, J. Assoc. Inf. Sci. Technol..

[8]  Per Ahlgren,et al.  The effects and their stability of field normalization baseline on relative performance with respect to citation impact: A case study of 20 natural science departments , 2011, J. Informetrics.

[9]  Loet Leydesdorff,et al.  Turning the tables in citation analysis one more time: Principles for comparing sets of documents by using an “Integrated Impact Indicator” (I3) , 2011 .

[10]  R. Rousseau Percentile rank scores are congruous indicators of relative performance, or aren't they? , 2011, 1108.1860.

[11]  Thed N. van Leeuwen,et al.  The Leiden ranking 2011/2012: Data collection, indicators, and interpretation , 2012, J. Assoc. Inf. Sci. Technol..

[12]  Loet Leydesdorff,et al.  Accounting for the Uncertainty in the Evaluation of Percentile Ranks , 2012, J. Assoc. Inf. Sci. Technol..

[13]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[14]  Loet Leydesdorff,et al.  The new Excellence Indicator in the World Report of the SCImago Institutions Rankings 2011 , 2011, J. Informetrics.

[15]  Graham Thornicroft,et al.  Fair assessment of the merits of psychiatric research , 2007, British Journal of Psychiatry.

[16]  Loet Leydesdorff,et al.  Percentile ranks and the integrated impact indicator (I3) , 2011, J. Assoc. Inf. Sci. Technol..

[17]  Alexander I. Pudovkin,et al.  Percentile Rank and Author Superiority Indexes for Evaluating Individual Journal Articles and the Author’s Overall Citation Performance , 2009 .

[18]  Michael Schreiber,et al.  Inconsistencies of recently proposed citation impact indicators and how to avoid them , 2012, J. Assoc. Inf. Sci. Technol..