Impact vitality: an indicator based on citing publications in search of excellent scientists

This paper contributes to the quest for an operational definition of ‘research excellence’ and proposes a translation of the excellence concept into a bibliometric indicator. Starting from a textual analysis of funding program calls aimed at individual researchers and from the challenges for an indicator at this level in particular, a new type of indicator is proposed. The impact vitality indicator (Rons and Amez, 2008) reflects the vitality of the impact of a researcher's publication output, based on the change in volume over time of the citing publications. The introduced metric is shown to possess attractive operational characteristics and meets a number of criteria which are desirable when comparing individual researchers. The validity of one of the possible indicator variants is tested using a small dataset of applicants for a senior full-time research fellowship. Options for further research involve testing various indicator variants on larger samples linked to different kinds of evaluations. Copyright , Beech Tree Publishing.

[1]  D. Chubin,et al.  Peerless Science: Peer Review and U. S. Science Policy , 1990 .

[2]  D. Aksnes,et al.  Peer reviews and bibliometric indicators: a comparative study at a Norwegian university , 2004 .

[3]  John Antonakis,et al.  Quantifying Scholarly Impact: IQp Versus the Hirsch h , 2008 .

[4]  Henk F. Moed,et al.  Citation Analysis in Research Evaluation , 1899 .

[5]  Liv Langfeldt,et al.  Expert panels evaluating research: decision-making and sources of bias , 2004 .

[6]  John Antonakis,et al.  Quantifying Scholarly Impact: IQp Versus the Hirsch h , 2008, J. Assoc. Inf. Sci. Technol..

[7]  Elizabeth Aversa,et al.  The web of knowledge: a festschrift in honor of Eugene Garfield , 2001 .

[8]  Per Ottar Seglen,et al.  The skewness of science , 1992 .

[9]  Nadine Rons,et al.  Reliability and Comparability of Peer Review Results , 2013, ArXiv.

[10]  Robert J. W. Tijssen,et al.  Scoreboards of research excellence , 2003 .

[11]  Anthony F. J. van Raan,et al.  Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods , 2005, Scientometrics.

[12]  Per O. Seglen,et al.  The Skewness of Science , 1992, J. Am. Soc. Inf. Sci..

[13]  Kevin W. Boyack,et al.  Generation of large-scale maps of science and associated indicators. , 2005 .

[14]  Thed N. van Leeuwen,et al.  The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence , 2003, Scientometrics.

[15]  Ed J. Rinia,et al.  COMPARATIVE ANALYSIS OF A SET OF BIBLIOMETRIC INDICATORS AND CENTRAL PEER REVIEW CRITERIA. EVALUATION OF CONDENSED MATTER PHYSICS IN THE NETHERLANDS , 1998 .

[16]  Michel Zitt,et al.  REVISITING COUNTRY AND INSTITUTION INDICATORS FROM CITATION DISTRIBUTIONS : PROFILE PERFORMANCE MEASURES , 2007 .

[17]  R. Rousseau,et al.  The R- and AR-indices: Complementing the h-index , 2007 .

[18]  Peder Olesen Larsen,et al.  Counting methods are decisive for rankings based on publication and citation studies , 2005, Scientometrics.

[19]  Liming Liang,et al.  h-index sequence and h-index matrix: Constructions and applications , 2006, Scientometrics.

[20]  Aparna Basu,et al.  Using ISI's 'Highly Cited Researchers' to obtain a country level indicator of citation excellence , 2006, Scientometrics.

[21]  Thed N. van Leeuwen,et al.  Benchmarking international scientific excellence: Are highly cited research papers an appropriate frame of reference? , 2002, Scientometrics.

[22]  J. R. Cole,et al.  Chance and consensus in peer review. , 1981, Science.

[23]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[24]  Kevin W. Boyack,et al.  Thought leadership: A new indicator for national and institutional comparison , 2008, Scientometrics.

[25]  Nadine Rons,et al.  Impact Vitality - A Measure for Excellent Scientists , 2013, ArXiv.

[26]  Wolfgang Glänzel,et al.  Better late than never? On the chance to become highly cited only beyond the standard bibliometric time horizon , 2004, Scientometrics.

[27]  Liv Langfeldt,et al.  The Decision-Making Constraints and Processes of Grant Peer Review, and Their Effects on the Review Outcome , 2001, Peer review in an Era of Evaluation.