Ranking national research systems by citation indicators. A comparative analysis using whole and fractionalised counting methods

This paper presents an empirical analysis of two different methodologies for calculating national citation indicators: whole counts and fractionalised counts. The aim of our study is to investigate the effect on relative citation indicators when citations to documents are fractionalised among the authoring countries. We have performed two analyses: a time series analysis of one country and a cross-sectional analysis of 23 countries. The results show that all countries’ relative citation indicators are lower when fractionalised counting is used. Further, the difference between whole and fractionalised counts is generally greatest for the countries with the highest proportion of internationally co-authored articles. In our view there are strong arguments in favour of using fractionalised counts to calculate relative citation indexes at the national level, rather than using whole counts, which is the most common practice today.

[1]  S. Merhar,et al.  Letter to the editor , 2005, IEEE Communications Magazine.

[2]  Guido Van Hooydonk.  Fractional Counting of Multiauthored Publications: Consequences for the Impact of Authors , 1997, J. Am. Soc. Inf. Sci..

[3]  Loet Leydesdorff,et al.  Problems with the ‘measurement’ of national scientific performance , 1988 .

[4]  P H Abelson.  President reagan, science, and engineering. , 1981, Science.

[5]  H. Small,et al.  Modifying the journal impact factor by fractional citation weighting: The audience factor , 2008 .

[6]  Guido Van Hooydonk,et al.  Fractional Counting of Multiauthored Publications: Consequences for the Impact of Authors , 1997, J. Am. Soc. Inf. Sci..

[7]  Dag W. Aksnes,et al.  A macro study of self-citation , 2003, Scientometrics.

[8]  H. Herbertz,et al.  Does it pay to cooperate? A bibliometric case study in molecular biology , 1995, Scientometrics.

[9]  Serge Galam,et al.  Tailor based allocations for multiple authorship: a fractional gh-index , 2010, Scientometrics.

[10]  Pedro Albarrán,et al.  The measurement of scientific excellence around the world , 2012 .

[11]  Daryl E. Chubin,et al.  Is citation analysis a legitimate evaluation tool? , 1979, Scientometrics.

[12]  M. Gunnarsson,et al.  International Research Cooperation in the Nordic Countries : A publication from the NORIA-net "The use of bibliometrics in research policy and evaluation activities" , 2010 .

[13]  Wolfgang Glänzel,et al.  Science in Scandinavia: A Bibliometric Approach , 2004, Scientometrics.

[14]  Loet Leydesdorff,et al.  Caveats for the journal and field normalizations in the CWTS ("Leiden") evaluations of research performance , 2010, J. Informetrics.

[15]  Tibor Braun,et al.  Relative indicators and relational charts for comparative assessment of publication output and citation impact , 1986, Scientometrics.

[16]  Amy M. Hightower,et al.  Science and Engineering Indicators , 1993 .

[17]  Peder Olesen Larsen,et al.  Counting methods are decisive for rankings based on publication and citation studies , 2005, Scientometrics.

[18]  Tibor Braun,et al.  Cross-field normalization of scientometric indicators , 1996, Scientometrics.

[19]  Thed N. van Leeuwen,et al.  New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications , 1995, Scientometrics.

[20]  Leo Egghe,et al.  Methods for accrediting publications to authors or countries: Consequences for evaluation studies , 2000, J. Am. Soc. Inf. Sci..

[21]  Terje Bruen Olsen,et al.  Validation of Bibliometric Indicators in the Field of Microbiology: A Norwegian Case Study , 2004, Scientometrics.

[22]  Michel Zitt,et al.  Citing-side normalization of journal impact: A robust variant of the Audience Factor , 2010, J. Informetrics.

[23]  Tibor Braun,et al.  AGAINST ABSOLUTE METHODS: RELATIVE SCIENTOMETRIC INDICATORS AND RELATIONAL CHARTS AS EVALUATION TOOLS , 1988 .

[24]  Martha A. Harsanyi.  Multiple authors, multiple problems: bibliometrics and the study of scholarly collaboration: a literature review , 1993 .

[25]  Ben R. Martin,et al.  British science in the 1980s — Has the relative decline continued? , 2005, Scientometrics.

[26]  Michel Zitt,et al.  Modifying the journal impact factor by fractional citation weighting: The audience factor , 2008, J. Assoc. Inf. Sci. Technol..

[27]  C. D. Hurt.  Conceptual citation differences in science, technology, and social sciences literature , 1987, Inf. Process. Manag..

[28]  Henk F. Moed,et al.  Citation Analysis in Research Evaluation , 1899 .

[29]  T. Palva,et al.  Pages 1-19 , 2001 .

[30]  Peter Vinkler,et al.  Evaluation of some methods for the relative assessment of scientific publications , 1986, Scientometrics.

[31]  Wolfgang Glänzel,et al.  Inflationary bibliometric values: The role of scientific collaboration and the need for relative indicators in evaluative studies , 2004, Scientometrics.

[32]  Jonas Lundberg,et al.  Lifting the crown - citation z-score , 2007, J. Informetrics.

[33]  Tim Peacock,et al.  Charting the decline in British science , 1985, Nature.

[34]  Jesper W. Schneider.  Bibliometric Research Performance Indicators for the Nordic Countries : A publication from the NORIA-net "The use of bibliometrics in research policy and evaluation activities" , 2010 .

[35]  Thed N. van Leeuwen,et al.  Towards a new crown indicator: Some theoretical considerations , 2010, J. Informetrics.

[36]  Anthony F. J. van Raan,et al.  The influence of international collaboration on the impact of research results , 1998, Scientometrics.

[37]  Loet Leydesdorff,et al.  of Science , 2022 .