Examples for counterintuitive behavior of the new citation-rank indicator P100 for bibliometric evaluations

A new percentile-based rating scale P100 has recently been proposed to describe the citation impact in terms of the distribution of the unique citation values. Here I investigate P100 for 5 example datasets, two simple fictitious models and three larger empirical samples. Counterintuitive behavior is demonstrated in the model datasets, pointing to difficulties when the evolution with time of the indicator is analyzed or when different fields or publication years are compared. It is shown that similar problems can occur for the three larger datasets of empirical citation values. Further, it is observed that the performance evaluation result in terms of percentiles can be influenced by selecting different journals for publication of a manuscript.

[1]  Lutz Bornmann,et al.  From P100 to P100': A new citation‐rank approach , 2014, J. Assoc. Inf. Sci. Technol..

[2]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[3]  Allen Hazen,et al.  Storage to be Provided Impounding Reservoirs for Municipal Water Supply , 1913 .

[4]  Loet Leydesdorff,et al.  The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits , 2012, J. Informetrics.

[5]  Sune Lehmann,et al.  A quantitative analysis of indicators of scientific performance , 2008, Scientometrics.

[6]  Michael Schreiber,et al.  Uncertainties and ambiguities in percentiles and how to avoid them , 2012, J. Assoc. Inf. Sci. Technol..

[7]  Lutz Bornmann,et al.  From P100 to P100_: Conception and improvement of a new citation-rank approach in bibliometrics , 2013, 1307.0667.

[8]  Lutz Bornmann,et al.  Redundancies in H Index Variants and the Proposal of the Number of Top-Cited Papers as an Attractive Indicator , 2012 .

[9]  Jian Wang,et al.  Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed one (P100) , 2013, ArXiv.

[10]  J. Hirsch.  Does the h index have predictive power? , 2007, Proceedings of the National Academy of Sciences.

[11]  Michael Schreiber,et al.  How much do different ways of calculating percentiles influence the derived performance indicators? A case study , 2013, Scientometrics.

[12]  Jian Wang,et al.  Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P100) , 2013, J. Informetrics.

[13]  Michael Schreiber.  Empirical evidence for the relevance of fractional scoring in the calculation of percentile rank scores , 2013, J. Assoc. Inf. Sci. Technol..

[14]  T. V. Leeuwen.  Testing the validity of the Hirsch-index for research assessment purposes , 2008 .