Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine

In this study, we examined empirical results on the h index and its most important variants in order to determine whether the variants developed are associated with an incremental contribution for evaluation purposes. The results of a factor analysis using bibliographic data on postdoctoral researchers in biomedicine indicate that regarding the h index and its variants, we are dealing with two types of indices that load on one factor each. One type describes the most productive core of a scientist's output and gives the number of papers in that core. The other type of indices describes the impact of the papers in the core. Because an index for evaluative purposes is a useful yardstick for comparison among scientists if the index corresponds strongly with peer assessments, we calculated a logistic regression analysis with the two factors resulting from the factor analysis as independent variables and peer assessment of the postdoctoral researchers as the dependent variable. The results of the regression analysis show that peer assessments can be predicted better using the factor ‘impact of the productive core’ than using the factor ‘quantity of the productive core.’ © 2008 Wiley Periodicals, Inc.

[1]  Yang Tao,et al.  A Study on Development Planning for Management Science and Engineering , 2006 .

[2]  Stevan Harnad,et al.  Open access scientometrics and the UK Research Assessment Exercise , 2007, Scientometrics.

[3]  Lutz Bornmann,et al.  Selection of research fellowship recipients by committee peer review. Reliability, fairness and predictive validity of Board of Trustees' decisions , 2005, Scientometrics.

[4]  W. Glänzel,et al.  A Hirsch-type index for journals [1] , 2005 .

[5]  Ronald Rousseau,et al.  Hirsch-type indices and library management: the case of Tongji University Library 1 , 2007 .

[6]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[7]  Wolfgang Glänzel,et al.  A Hirsch-type index for journals , 2006, Scientometrics.

[8]  L. Egghe How to improve the h-index , 2006 .

[9]  M. Kosmulski A new Hirsch-type index saves time and works equally well as the original h-index , 2009 .

[10]  Tünde Gracza,et al.  [Impact factor and/or Hirsch index?]. , 2007, Orvosi hetilap.

[11]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[12]  Lutz Bornmann,et al.  Does the h-index for ranking of scientists really work? , 2005, Scientometrics.

[13]  Juan Imperial,et al.  Usefulness of Hirsch’s h-index to evaluate scientific research in Spain , 2007, Scientometrics.

[14]  Leo Egghe,et al.  The power of power laws and an interpretation of Lotkaian informetric systems as self-similar fractals: Research Articles , 2005 .

[15]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[16]  Ronald Rousseau,et al.  New developments related to the Hirsch index , 2006 .

[17]  Lutz Bornmann,et al.  What do we know about the h index? , 2007, J. Assoc. Inf. Sci. Technol..

[18]  S. West,et al.  The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. , 1996 .

[19]  Bihui Jin The AR-index: complementing the h-index , 2007 .

[20]  D. Hosmer,et al.  Applied Logistic Regression , 1991 .

[21]  S. Cole Citations and the evaluation of individual scientists , 1989 .

[23]  David E. Booth,et al.  Applied Multivariate Analysis , 2003, Technometrics.

[24]  J. Hirsch Does the h index have predictive power? , 2007, Proceedings of the National Academy of Sciences.

[25]  L. Egghe,et al.  The power of power laws and an interpretation of Lotkaian informetric systems as self-similar fractals , 2005, J. Assoc. Inf. Sci. Technol..

[26]  Yannis Manolopoulos,et al.  Generalized Hirsch h-index for disclosing latent facts in citation networks , 2007, Scientometrics.

[27]  Lutz Bornmann,et al.  Convergent validation of peer review decisions using the h index: Extent of and reasons for type I and type II errors , 2007, J. Informetrics.

[28]  Lutz Bornmann,et al.  Selecting scientific excellence through committee peer review - A citation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicants , 2006, Scientometrics.

[29]  R. Rousseau,et al.  The R- and AR-indices: Complementing the h-index , 2007 .

[30]  S. Brooks,et al.  Applied Multivariate Statistics for the Social Sciences , 1993 .

[31]  P. Kline The New Psychometrics: Science, Psychology and Measurement , 1998 .

[32]  Rex B. Kline,et al.  Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research , 2004 .

[33]  Wolfgang Glänzel,et al.  On the h-index - A mathematical approach to a new measure of publication activity and citation impact , 2006, Scientometrics.

[34]  I. Bernstein Applied Multivariate Analysis , 1988 .

[35]  Juan E. Iglesias,et al.  Scaling the h-index for different scientific ISI fields , 2006, Scientometrics.

[36]  Philip Ball,et al.  Index aims for fair ranking of scientists , 2005, Nature.

[37]  Quentin L. Burrell,et al.  Hirsch's h-index: A stochastic model , 2007, J. Informetrics.

[38]  L. Egghe An improvement of the h-index: the g-index , 2006 .

[39]  Quentin L. Burrell,et al.  On the h-index, the size of the Hirsch core and Jin's A-index , 2007, J. Informetrics.

[40]  Data Point , 2005, Science.

[41]  R. Gonzalez Applied Multivariate Statistics for the Social Sciences , 2003 .

[42]  M. Jennions,et al.  The h index and career assessment by numbers. , 2006, Trends in ecology & evolution.

[43]  Mônica G. Campiteli,et al.  Is it possible to compare researchers with different scientific interests? , 2006, Scientometrics.

[44]  L. Egghe,et al.  Theory and practise of the g-index , 2006, Scientometrics.

[45]  Philip Ball,et al.  Achievement index climbs the ranks , 2007, Nature.

[46]  Lokman I. Meho The Rise and Rise of Citation Analysis , 2006, ArXiv.

[47]  J. Vanclay Refining the H-Index , 2006 .

[48]  Michael Schreiber,et al.  A case study of the Hirsch index for 26 non‐prominent physicists , 2007, 0708.0120.

[49]  Leo Egghe,et al.  An h-index weighted by citation impact , 2008, Inf. Process. Manag..