Comparison of Brazilian researchers in clinical medicine: are criteria for ranking well-adjusted?

Quantifying the relative performance of individual scholars has become an integral part of decision-making in research policy. The objective of the present study was to evaluate if the scholarship rank of Brazilian Council for Scientific and Technological Development (CNPq) researchers in Medicine is consistent with their scientific productivity. The Lattes curricula of 411 researchers (2006–2008) were included in the study. Scholarship category was the variable of interest. Other variables analyzed were: time since receiving the doctorate, teaching activity (undergraduate, master’s and doctoral students), number of articles published, and number of papers indexed by the Institute for Scientific Information (ISI) and Scopus databases. Additional performance indicators included were: citations, h-index, and m-index. There was a significant difference among scholarship categories regarding number of papers per year, considering the entire scientific career (P < 0.001) or the last 5 years (P < 0.001). There was no significant difference among scholarship categories regarding the number of citations per article in the ISI (Thomson Reuters) database (P = 0.23). There was a significant difference in h-index among scholarship categories in both databases, i.e. (P < 0.001) and Scopus (P < 0.001). Regarding the m-index, there was a significant difference among categories only in the ISI database (P = 0.012). According to our findings, a better instrument for qualitative and quantitative indicators is needed to identify researchers with outstanding scientific output.

[1]  Quentin L. Burrell,et al.  Hirsch index or Hirsch rate? Some thoughts arising from Liang’s data , 2007, Scientometrics.

[2]  Lutz Bornmann,et al.  Is the h index related to (standard) bibliometric measures and to the assessments by peers? An investigation of the h index by using molecular life sciences data , 2008 .

[3]  Suelleng Maria Cunha Santos,et al.  Perfil dos pesquisadores da Saúde Coletiva no Conselho Nacional de Desenvolvimento Científico e Tecnológico , 2009 .

[4]  A. Ribeiro,et al.  Profile and scientific production of CNPq researchers in cardiology. , 2011, Arquivos brasileiros de cardiologia.

[5]  Giovanni Abramo,et al.  Research productivity: Are higher academic ranks more productive than lower ones? , 2011, Scientometrics.

[6]  R. Pecoits-Filho,et al.  Profile and scientific production of CNPq researchers in Nephrology and Urology. , 2011, Jornal brasileiro de nefrologia : 'orgao oficial de Sociedades Brasileira e Latino-Americana de Nefrologia.

[7]  Giovanni Abramo,et al.  National research assessment exercises: the effects of changing the rules of the game during the game , 2011, Scientometrics.

[8]  Anthony F. J. van Raan.  Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups , 2013, Scientometrics.

[9]  Wolfgang Glänzel,et al.  On the h-index - A mathematical approach to a new measure of publication activity and citation impact , 2006, Scientometrics.

[10]  A. D. Jackson,et al.  Measures for measures , 2006, Nature.

[11]  M. Goldbaum,et al.  Perfil dos pesquisadores com bolsa de produtividade em pesquisa do CNPq da área de saúde coletiva , 2003 .

[12]  Nicole Haeffner-Cavaillon,et al.  The use of bibliometric indicators to help peer-review assessment , 2009, Archivum Immunologiae et Therapiae Experimentalis.

[13]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[14]  Raika Augusta Cavalcante,et al.  Perfil dos pesquisadores da área de odontologia no Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) , 2008 .

[15]  H. Browman,et al.  Factors and indices are one thing, deciding who is scholarly, why they are scholarly, and the relative value of their scholarship is something else entirely , 2008 .

[16]  Rogério Mugnaini,et al.  A new indicator for international visibility: exploring Brazilian scientific community , 2011, Scientometrics.

[17]  Quentin L. Burrell,et al.  Hirsch's h-index: A stochastic model , 2007, J. Informetrics.

[18]  Sotaro Shibayama,et al.  Distribution of academic research funds: a case of Japanese national research grant , 2011, Scientometrics.

[19]  Lutz Bornmann,et al.  OPEN PEN ACCESS CCESS , 2008 .

[20]  M. Goldbaum,et al.  [A profile of researchers in public health with productivity grants from the Brazilian National Research Council (CNPq)]. , 2003, Cadernos de saude publica.

[21]  Lutz Bornmann,et al.  Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine , 2008, J. Assoc. Inf. Sci. Technol..

[22]  LetaJacqueline,et al.  A new indicator for international visibility , 2011 .

[23]  Kuan-Teh Jeang,et al.  H-index, mentoring-index, highly-cited and highly-accessed: how to evaluate scientists? , 2008, Retrovirology.

[24]  Sebastian K. Boell,et al.  Journal Impact Factors for evaluating scientific performance: use of h-like indicators , 2010, Scientometrics.

[25]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[26]  Fiorenzo Franceschini,et al.  Proposals for evaluating the regularity of a scientist’s research output , 2011, Scientometrics.

[27]  Lutz Bornmann,et al.  What do we know about the h index? , 2007, J. Assoc. Inf. Sci. Technol..

[28]  Jacques Wainer,et al.  Brazilian computer science research: Gender and regional distributions , 2009, Scientometrics.

[29]  Wolfgang Glänzel,et al.  A new classification scheme of science fields and subfields designed for scientometric evaluation purposes , 2004, Scientometrics.

[30]  J. Lane.  Let's make science metrics more scientific , 2010, Nature.

[31]  Henk F. Moed,et al.  New developments in the use of citation analysis in research evaluation , 2009, Archivum Immunologiae et Therapiae Experimentalis.

[32]  Natacha Carvalho Ferreira Santos,et al.  Produtividade em pesquisa do CNPq: análise do perfil dos pesquisadores da Química , 2010 .

[33]  B. Martin,et al.  University Research Evaluation and Funding: An International Comparison , 2003 .

[34]  M. Walport,et al.  Looking for Landmarks: The Role of Expert Review and Bibliometric Analysis in Evaluating Scientific Publication Outputs , 2009, PloS one.

[35]  A. Regalado,et al.  Brazilian Science: Riding a Gusher , 2010 .

[36]  John Panaretos,et al.  Assessing scientific research performance and impact with single indices , 2008, Scientometrics.

[37]  A. Petherick.  High hopes for Brazilian science , 2010, Nature.

[38]  Igor Podlubny,et al.  Comparison of scientific impact expressed by the number of citations in different fields of science , 2004, Scientometrics.

[39]  P. Mendes,et al.  Perfil dos pesquisadores bolsistas de produtividade científica em medicina no CNPq, Brasil , 2010 .

[40]  I. Quirino,et al.  Pesquisadores do CNPq na área de medicina: comparação das áreas de atuação , 2010 .