The inconsistency of the h-index

The h-index is a popular bibliometric indicator for assessing individual scientists. We criticize the h-index from a theoretical point of view. We argue that for the purpose of measuring the overall scientific impact of a scientist (or some other unit of analysis), the h-index behaves in a counterintuitive way. In certain cases, the mechanism used by the h-index to aggregate publication and citation statistics into a single number leads to inconsistencies in the way in which scientists are ranked. Our conclusion is that the h-index cannot be considered an appropriate indicator of a scientist's overall scientific impact. Based on recent theoretical insights, we discuss what kind of indicators can be used as an alternative to the h-index. We pay special attention to the highly cited publications indicator. This indicator has a lot in common with the h-index, but unlike the h-index it does not produce inconsistent rankings. © 2012 Wiley Periodicals, Inc.

[1]  Leo Egghe,et al.  Characterizations of the generalized Wu- and Kosmulski-indices in Lotkaian systems , 2011, J. Informetrics.

[2]  Jonas Lundberg,et al.  Lifting the crown - citation z-score , 2007, J. Informetrics.

[3]  Reinier Plomp,et al.  The highly cited papers of professors as an indicator of a research group's scientific performance , 1994, Scientometrics.

[4]  Wolfgang Glänzel,et al.  On the h-index - A mathematical approach to a new measure of publication activity and citation impact , 2006, Scientometrics.

[5]  A. D. Jackson,et al.  Measures for measures , 2006, Nature.

[6]  Ludo Waltman,et al.  Generalizing the H- and G-Indices , 2008, J. Informetrics.

[7]  Jorge E. Hirsch,et al.  An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship , 2009, Scientometrics.

[8]  Thierry Marchant.  Score-based bibliometric rankings of authors , 2009 .

[9]  M. Kosmulski.  A new Hirsch-type index saves time and works equally well as the original h-index , 2009 .

[10]  Wolfgang Glänzel,et al.  A Hirsch-type index for journals , 2006, Scientometrics.

[11]  Martin Ravallion,et al.  On measuring scholarly influence by citations , 2011, Scientometrics.

[12]  Anthony F. J. van Raan.  Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups , 2013, Scientometrics.

[13]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[14]  Thed N. van Leeuwen,et al.  Towards a new crown indicator: Some theoretical considerations , 2010, J. Informetrics.

[15]  J. Hirsch.  Does the h index have predictive power? , 2007, Proceedings of the National Academy of Sciences.

[16]  Gerhard J. Woeginger,et al.  An axiomatic characterization of the Hirsch-index , 2008, Math. Soc. Sci..

[17]  Thierry Marchant,et al.  Bibliometric rankings of journals based on Impact Factors: An axiomatic approach , 2011, J. Informetrics.

[18]  Michael Schreiber,et al.  A modification of the h-index: The hm-index accounts for multi-authored manuscripts , 2008, J. Informetrics.

[19]  Thierry Marchant,et al.  An axiomatic characterization of the ranking based on the h-index and some other bibliometric rankings of authors , 2009, Scientometrics.

[20]  Javier Ruiz-Castillo,et al.  The Measurement of Low- and High-Impact in Citation Distributions: Technical Results , 2011, J. Informetrics.

[21]  Ton van Raan.  Bibliometrics: Measure for measure , 2010, Nature.

[22]  Francisco Herrera,et al.  h-Index: A review focused in its variants, computation and standardization for different scientific fields , 2009, J. Informetrics.

[23]  Ludo Waltman,et al.  A Taxonomy of Bibliometric Performance Indicators Based on the Property of Consistency , 2009 .

[24]  Leo Egghe,et al.  Mathematical theory of the h- and g-index in case of fractional counting of authorship , 2008, J. Assoc. Inf. Sci. Technol..

[25]  Leo Egghe,et al.  An informetric model for the Hirsch-index , 2006, Scientometrics.

[26]  Wolfgang Glänzel,et al.  The diffusion of H-related literature , 2011, J. Informetrics.

[27]  Quentin L. Burrell,et al.  Hirsch's h-index: A stochastic model , 2007, J. Informetrics.

[28]  Michael Schreiber,et al.  Twenty Hirsch index variants and other indicators giving more or less preference to highly cited papers , 2010, ArXiv.

[29]  Reinier Plomp,et al.  The significance of the number of highly cited papers as an indicator of scientific prolificacy , 1990, Scientometrics.

[30]  Antonio Quesada,et al.  More axiomatics for the Hirsch index , 2010, Scientometrics.

[31]  J. Hirsch.  An index to quantify an individual's scientific output , 2005 .

[32]  Gerhard J. Woeginger,et al.  A new family of scientific impact measures: The generalized Kosmulski-indices , 2009, Scientometrics.

[33]  B. Mérialdo,et al.  THEORY AND PRACTISE , 1998 .

[34]  Nees Jan van Eck,et al.  A Simple Alternative to the H-Index , 2009 .

[35]  Rodrigo Costas,et al.  The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level , 2007, J. Informetrics.

[36]  Leo Egghe,et al.  The Hirsch index and related impact measures , 2010, Annu. Rev. Inf. Sci. Technol..

[37]  Lutz Bornmann,et al.  A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants , 2011, J. Informetrics.

[38]  Qiang Wu,et al.  The w-index: A measure to assess scientific impact by focusing on widely cited papers , 2010 .

[39]  Thierry Marchant,et al.  Ranking scientists and departments in a consistent manner , 2011, J. Assoc. Inf. Sci. Technol..

[40]  Glenn Ellison,et al.  How Does the Market Use Citation Data? The Hirsch Index in Economics , 2010, SSRN Electronic Journal.

[41]  Thed N. van Leeuwen,et al.  Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference? , 2002, Scientometrics.

[42]  Sune Lehmann,et al.  A quantitative analysis of indicators of scientific performance , 2008, Scientometrics.

[43]  Oscar Volij,et al.  The Measurement of Intellectual Influence , 2002 .

[44]  L. Egghe,et al.  Theory and practise of the g-index , 2006, Scientometrics.

[45]  Lutz Bornmann,et al.  What do we know about the h index? , 2007, J. Assoc. Inf. Sci. Technol..

[46]  Qiang Wu,et al.  The w-index: A measure to assess scientific impact by focusing on widely cited papers , 2010, J. Assoc. Inf. Sci. Technol..

[47]  Nicola De Bellis,et al.  The Publish or Perish Book , 2011, J. Assoc. Inf. Sci. Technol..

[48]  Charles Oppenheim,et al.  The h-index: a broad review of a new bibliometric indicator , 2010, J. Documentation.

[49]  Ludo Waltman,et al.  The relation between Eigenfactor, audience factor, and influence weight , 2010, J. Assoc. Inf. Sci. Technol..