The journal impact factor: angel, devil, or scapegoat? A comment on J.K. Vanclay’s article 2011

J.K. Vanclay’s article is a bold attempt to review recent works on the journal impact factor (JIF) and to call for alternative certifications of journals. The too broad scope did not allow the author to fulfill all his purposes. Attempting after many others to organize the various forms of criticism, with targets often broader than the JIF, we shall try to comment on a few points. This will hopefully enable us to infer in which cases the JIF is an angel, a devil, or a scapegoat. We shall also expand on a crucial question that Vanclay could not really develop in the reduced article format: the field-normalization. After a short recall on classical cited-side or ex post normalization and of the powerful influence measures, we will devote some attention to the novel way of citing-side or ex ante normalization, not only for its own interest, but because it directly proceeds from the disassembling of the JIF clockwork.

[1]  Michel Zitt,et al.  Correcting glasses help fair comparisons in international science landscape: Country indicators as a function of ISI database delineation , 2003, Scientometrics.

[2]  Jerome K. Vanclay,et al.  Impact factor: outdated artefact or stepping-stone to journal certification? , 2011, Scientometrics.

[3]  E. Garfield,et al.  Citation indexes for science. , 1956, Science.

[4]  Judit Bar-Ilan,et al.  Which h-index? — A comparison of WoS, Scopus and Google Scholar , 2008, Scientometrics.

[5]  Mark Abrahamson,et al.  The Scientific Community. , 1966 .

[6]  Ronald Rousseau,et al.  Aggregation properties of relative impact and other classical indicators: Convexity issues and the Yule-Simpson paradox , 2009, Scientometrics.

[7]  K. Knorr-Cetina,et al.  Advances in social theory and methodology : toward an integration of micro- and macro-sociologies , 1981 .

[8]  Per O. Seglen,et al.  The Skewness of Science , 1992, J. Am. Soc. Inf. Sci..

[9]  Eugene Garfield,et al.  New factors in the evaluation of scientific literature through citation indexing , 1963 .

[10]  Henry G. Small,et al.  Clustering thescience citation index® using co-citations , 1985, Scientometrics.

[11]  Henk F. Moed,et al.  Possible inaccuracies occurring in citation analysis , 1989, J. Inf. Sci..

[12]  Tove Faber Frandsen,et al.  The Reference Return Ratio , 2008, J. Informetrics.

[13]  Henk F. Moed,et al.  Improving the accuracy of Institute for Scientific Information's journal impact factors , 1995 .

[14]  Henk F. Moed,et al.  Journal impact measures in bibliometric research , 2004, Scientometrics.

[15]  A. Remhof,et al.  Towards a new , 1997 .

[16]  Michel Zitt,et al.  Relativity of citation performance and excellence measures: From cross-field to cross-scale effects of field-normalisation , 2005, Scientometrics.

[17]  P. Bourdieu.  The specificity of the scientific field and the social conditions of the progress of reason , 1975 .

[18]  Jan M. Rabaey,et al.  Comparison of Methods , 2004 .

[19]  Gabriel Pinski,et al.  Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics , 1976, Inf. Process. Manag..

[20]  K. K. Cetina,et al.  Unscrewing the big Leviathan: how actors macro- structure reality and how sociologists help them to do so , 2014 .

[21]  Wolfgang Glänzel,et al.  A Hirsch-type index for journals , 2006, Scientometrics.

[22]  B. K. Sen.  Normalised Impact factor , 1992, J. Documentation.

[23]  Wolfgang Glänzel,et al.  On some new bibliometric applications of statistics related to the h-index , 2008, Scientometrics.

[24]  Ronald Rousseau,et al.  Journal of Informetrics -index , 2022 .

[25]  Peter Vinkler,et al.  Subfield problems in applying the Garfield (Impact) Factors in practice , 2002, Scientometrics.

[26]  Anthony F. J. van Raan,et al.  Competition amongst scientists for publication status:Toward a model of scientific publication and citation distributions , 2001, Scientometrics.

[27]  J. S. Katz,et al.  The self-similar science system , 1999 .

[28]  Claudio Castellano,et al.  Universality of citation distributions: Toward an objective measure of scientific impact , 2008, Proceedings of the National Academy of Sciences.

[29]  Gideon Czapski,et al.  The use of deciles of the citation impact to evaluate different fields of research in Israel , 1997, Scientometrics.

[30]  Anthony F. J. van Raan,et al.  On Growth, Ageing, and Fractal Differentiation of Science , 2000, Scientometrics.

[31]  J. Hirsch.  Does the h index have predictive power? , 2007, Proceedings of the National Academy of Sciences.

[32]  Carl T. Bergstrom.  Eigenfactor Measuring the value and prestige of scholarly journals , 2007 .

[33]  Henk F. Moed,et al.  Measuring contextual citation impact of scientific journals , 2009, J. Informetrics.

[34]  Nancy L. Geller,et al.  On the citation influence methodology of Pinski and Narin , 1978, Inf. Process. Manag..

[35]  Michael J. Moravcsik,et al.  Variation of the nature of citation measures with journals and scientific specialties , 1978, J. Am. Soc. Inf. Sci..

[36]  Jonas Lundberg,et al.  Lifting the crown - citation z-score , 2007, J. Informetrics.

[37]  L. Zucker,et al.  Star scientists and institutional transformation: patterns of invention and innovation in the formation of the biotechnology industry. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[38]  D. Price.  Little Science, Big Science , 1965 .

[39]  Oscar Volij,et al.  The Measurement of Intellectual Influence , 2002 .

[40]  C. Hoeffel,et al.  journal impact factors , 1998, Allergy.

[41]  D. Hicks.  The Four Literatures of Social Science , 2004 .

[42]  Irina Marshakova-Shaik Evich.  THE STANDARD IMPACT FACTOR AS AN EVALUATION TOOL OF SCIENCE FIELDS AND SCIENTIFIC JOURNALS , 1996 .

[43]  Terttu Luukkonen,et al.  Why has Latour's theory of citations been ignored by the bibliometric community? discussion of sociological interpretations of citation analysis , 2006, Scientometrics.

[44]  Blaise Cronin,et al.  The citation process: The role and significance of citations in scientific communication , 1984 .

[45]  Thed N. van Leeuwen,et al.  Towards a new crown indicator: Some theoretical considerations , 2010, J. Informetrics.

[46]  Paul Wouters,et al.  Citation cycles and peer review cycles , 2006, Scientometrics.

[47]  Johan Bollen,et al.  A Principal Component Analysis of 39 Scientific Impact Measures , 2009, PloS one.

[48]  Norman Kaplan,et al.  The Sociology of Science: Theoretical and Empirical Investigations , 1974 .

[49]  Koenraad Debackere,et al.  A priori vs. a posteriori normalisation of citation indicators. The case of journal ranking , 2011, Scientometrics.

[50]  E. Garfield.  Citation analysis as a tool in journal evaluation. , 1972, Science.

[51]  Elizabeth S. Vieira,et al.  The journal relative impact: an indicator for journal assessment , 2011, Scientometrics.

[52]  E GARFIELD,et al.  Citation indexes for science; a new dimension in documentation through association of ideas. , 2006, Science.

[53]  Ludo Waltman,et al.  A Taxonomy of Bibliometric Performance Indicators Based on the Property of Consistency , 2009 .

[54]  E. Garfield.  The history and meaning of the journal impact factor. , 2006, JAMA.

[55]  Gabor Pataki,et al.  A Principal Component Analysis for Trees , 2008, 0810.0944.

[56]  Peter Ingwersen,et al.  The publication-citation matrix and its derived quantities , 2001 .

[57]  Michel Zitt,et al.  Behind citing-side normalization of citations: some properties of the journal impact factor , 2011, Scientometrics.

[58]  Michel Zitt,et al.  Modifying the journal impact factor by fractional citation weighting: The audience factor , 2008, J. Assoc. Inf. Sci. Technol..

[59]  Leo Egghe,et al.  A general framework for relative impact indicators , 2003 .

[60]  Michel Zitt,et al.  Citing-side normalization of journal impact: A robust variant of the Audience Factor , 2010, J. Informetrics.

[61]  Jerome K. Vanclay,et al.  Bias in the journal impact factor , 2006, Scientometrics.

[62]  Johannes Hönekopp,et al.  Future publication success in science is better predicted by traditional measures than by the h index , 2011, Scientometrics.

[63]  Jonathan Adams,et al.  Profiling citation impact: A new methodology , 2007, Scientometrics.

[64]  Henk F. Moed,et al.  Handbook of Quantitative Science and Technology Research , 2005 .

[65]  Thierry Marchant,et al.  Bibliometric rankings of journals based on Impact Factors: An axiomatic approach , 2011, J. Informetrics.

[66]  Thierry Marchant,et al.  An axiomatic characterization of the ranking based on the h-index and some other bibliometric rankings of authors , 2009, Scientometrics.

[67]  Peter Taylor,et al.  Citation Statistics , 2009, ArXiv.

[68]  Tibor Braun,et al.  Relative indicators and relational charts for comparative assessment of publication output and citation impact , 1986, Scientometrics.