Enhancing the h index for the objective assessment of healthcare researcher performance and impact

Summary Objective To investigate whether the h index (a bibliometric tool which is increasingly used to assess and appraise an individual's research performance) could be improved to better measure the academic performance and citation profile for individual healthcare researchers. Design Cohort study. Setting Faculty of Medicine, Imperial College London, UK. Participants Publication lists from 1 January 2000 until 31 December 2009 for 501 academic healthcare researchers from the Faculty of Medicine. Main outcome measures The h index for each researcher was calculated over a nine-year period. The citation count for each researcher was differentiated into high (h2 upper), core (h2 centre) and low (h2 lower) visibility areas. Segmented regression model (sRM) was used to statistically estimate number of high visibility publications (sRM value). Validity of the h index and other proposed adjuncts were analysed against academic rank and conventional bibliometric indicators. Results Construct validity was demonstrated for h index, h2 upper, h2 centre, h2 lower and sRM value (all P < 0.05). Convergent validity of the h index and sRM value was shown by significant correlations with total number of publications (r =0.89 and 0.86 respectively, P<0.05) and total number of citations (r = 0.96 and 0.65, respectively, P<0.05). Significant differences in h index and sRM value existed between non-physician and physician researchers (P < 0.05). Conclusions This study supports the construct validity of the h index as a measure of healthcare researcher academic rank. It also identifies the assessment value of our developed indices of h2 upper, h2 centre, h2 lower and sRM. These can be applied in combination with the h index to provide additional objective evidence to appraise the performance and impact of an academic healthcare researcher.

[1]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[2]  D. Ruppert Robust Statistics: The Approach Based on Influence Functions , 1987 .

[3]  Ara Darzi,et al.  Measuring Academic Performance for Healthcare Researchers with the H Index: Which Search Tool Should Be Used? , 2012, Medical Principles and Practice.

[4]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[5]  J. R. Cole,et al.  Scientific output and recognition: a study in the operation of the reward system in science. , 1967, American sociological review.

[6]  S. Bhayani,et al.  Does citation analysis reveal association between h-index and academic rank in urology? , 2009, Urology.

[7]  J. Hirsch An index to quantify an individual's scientific output , 2005 .

[8]  A. Kulkarni,et al.  Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. , 2009, JAMA.

[9]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[10]  John P. A. Ioannidis,et al.  More time for research: Fund people not projects , 2011, Nature.

[11]  Lutz Bornmann,et al.  The h index research output measurement: Two approaches to enhance its accuracy , 2010, J. Informetrics.

[12]  J. Wilberger,et al.  Survey of the h index for all of academic neurosurgery: another power-law phenomenon? , 2010, Journal of neurosurgery.

[13]  T. Grantcharov,et al.  h-Indices in a university department of anaesthesia: an evaluation of their feasibility, reliability, and validity as an assessment of academic performance. , 2011, British journal of anaesthesia.

[14]  R. Gonzalez Applied Multivariate Statistics for the Social Sciences , 2003 .

[15]  L. Bornmann,et al.  The state of h index research , 2009, EMBO reports.

[16]  Ara Darzi,et al.  Benchmarking in surgical research , 2009, The Lancet.

[17]  S. Durso,et al.  Implications of Academic Medicine’s Failure to Recognize Clinical Excellence , 2009, Clinical Medicine & Research.

[18]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[19]  H. Ashrafian,et al.  Nobel Prizes in Medicine: are clinicians out of fashion? , 2011, Journal of the Royal Society of Medicine.

[20]  Lutz Bornmann,et al.  A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants , 2011, J. Informetrics.

[21]  David F Kallmes,et al.  The H-index in academic radiology. , 2010, Academic radiology.

[22]  W. Couldwell,et al.  Use of the h index in neurosurgery. Clinical article. , 2009, Journal of neurosurgery.

[23]  A. Darzi,et al.  What does leadership in surgery entail? , 2010, ANZ journal of surgery.

[24]  Ara Darzi,et al.  How has healthcare research performance been assessed? A systematic review , 2011, Journal of the Royal Society of Medicine.

[25]  Lutz Bornmann,et al.  What do citation counts measure? A review of studies on citing behavior , 2008, J. Documentation.

[26]  Claudio Castellano,et al.  Universality of citation distributions: Toward an objective measure of scientific impact , 2008, Proceedings of the National Academy of Sciences.

[27]  D. Sheridan Reversing the decline of academic medicine in Europe , 2006, The Lancet.