What proportion of excellent papers makes an institution one of the best worldwide? Specifying thresholds for the interpretation of the results of the SCImago Institutions Ranking and the Leiden Ranking

University rankings generally present users with the problem of placing the results given for an institution in context. Only a comparison with the performance of all other institutions makes it possible to say exactly where an institution stands. In order to interpret the results of the SCImago Institutions Ranking (based on Scopus data) and the Leiden Ranking (based on Web of Science data), in this study we offer thresholds with which it is possible to assess whether an institution belongs to the top 1%, top 5%, top 10%, top 25%, or top 50% of institutions in the world. The thresholds are based on the excellence rate or PPtop 10%. Both indicators measure the proportion of an institution's publications which belong to the 10% most frequently cited publications and are the most important indicators for measuring institutional impact. For example, while an institution must achieve a value of 24.63% in the Leiden Ranking 2013 to be considered one of the top 1% of institutions worldwide, the SCImago Institutions Ranking requires 30.2%.

[1]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[2]  Giovanni Abramo,et al.  Testing the trade-off between productivity and quality in research activities , 2018, J. Assoc. Inf. Sci. Technol..

[3]  Amy M. Hightower,et al.  Science and Engineering Indicators , 1993 .

[4]  Thed N. van Leeuwen,et al.  The Leiden ranking 2011/2012: Data collection, indicators, and interpretation , 2012, J. Assoc. Inf. Sci. Technol..

[5]  Loet Leydesdorff,et al.  Turning the tables in citation analysis one more time: Principles for comparing sets of documents by using an “Integrated Impact Indicator” (I3) , 2011 .

[6]  Lutz Bornmann,et al.  A multilevel modelling approach to investigating the predictive validity of editorial decisions: do the editors of a high profile journal select manuscripts that are highly cited after publication? , 2011 .

[7]  Lutz Bornmann.  Assigning publications to multiple subject categories for bibliometric analysis: An empirical case study based on percentiles , 2014, J. Documentation.

[8]  Luk Arbuckle,et al.  Two h-Index Benchmarks for Evaluating the Publication Performance of Medical Informatics Researchers , 2012, Journal of medical Internet research.

[9]  Vicente P. Guerrero-Bote,et al.  A further step forward in measuring journals' scientific prestige: The SJR2 indicator , 2012, J. Informetrics.

[10]  Andrea Bergmann,et al.  Citation Indexing Its Theory And Application In Science Technology And Humanities , 2016 .

[11]  Lutz Bornmann,et al.  Some interesting insights from aggregated data published in the World Report SIR 2010 , 2011, J. Informetrics.

[12]  Va Arlington.  National Science Board. , 2010 .

[13]  Loet Leydesdorff,et al.  The new Excellence Indicator in the World Report of the SCImago Institutions Rankings 2011 , 2011, J. Informetrics.

[14]  Thed N. van Leeuwen,et al.  Towards a new crown indicator: Some theoretical considerations , 2010, J. Informetrics.

[15]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[16]  Sven Hemlin,et al.  Research on research evaluation , 1996 .

[17]  Ludo Waltman,et al.  On the calculation of percentile-based bibliometric indicators , 2012, J. Assoc. Inf. Sci. Technol..

[18]  Pedro Albarrán,et al.  The skewness of science in 219 sub-fields and a number of aggregates , 2010, Scientometrics.

[19]  Kevin W Boyack,et al.  Mapping knowledge domains: Characterizing PNAS , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[20]  L. Bornmann,et al.  How good is research really? , 2013, EMBO reports.

[21]  Lutz Bornmann,et al.  How to analyze percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes, and top-cited papers , 2013, J. Assoc. Inf. Sci. Technol..

[22]  Gabriel Kreiman,et al.  Nine Criteria for a Measure of Scientific Output , 2011, Front. Comput. Neurosci..

[23]  B. Jay Coleman,et al.  Benchmarking Individual Publication Productivity in Logistics , 2012 .

[24]  Loet Leydesdorff,et al.  The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits , 2012, J. Informetrics.