Letter to the Editor: About the quality and impact of scientific articles

It is argued that counting the total number of times a scientific article is cited by others, does neither result in a proxy for its cognitive impact nor for its quality. One would have to distinguish at least substitutable and fundamental references. A supposed correlation between peer review assessments and citation counts is conceptually problematic, because peer review includes objective as well as subjective considerations (convictions). With refined methods, however, a differential citation analysis might be able in the future to answer if a given article did or did not have positive cognitive impact on subsequent research.

[1]  Martin Ricker,et al.  MEASuRINg SCIENTISTS' PER fORMANCE: A vIEW fROM ORgANISMAl bIOlOgISTS , 2009 .

[2]  Alan Singleton,et al.  Bibliometrics and Citation Analysis; from the Science Citation Index to Cybermetrics , 2010, Learn. Publ..

[3]  Ulysses Paulino Albuquerque,et al.  Citation behavior in popular scientific papers: what is behind obscure citations? The case of ethnobotany , 2012, Scientometrics.

[4]  Vwani P. Roychowdhury,et al.  Stochastic modeling of citation slips , 2004, Scientometrics.

[5]  Ludo Waltman,et al.  Counting publications and citations: Is more always better? , 2013, J. Informetrics.

[6]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[7]  Sean Gerrish,et al.  A Language-based Approach to Measuring Scholarly Impact , 2010, ICML.

[8]  William J. Sutherland,et al.  Quantifying the Impact and Relevance of Scientific Research , 2011, PloS one.

[9]  C. Nicolini,et al.  Can the assignment of university chairs be automated? , 2005, Scientometrics.

[10]  Deborah Zornes,et al.  Defining and assessing research quality in a transdisciplinary context , 2016 .

[11]  Terry Speed.  Statistics for Experimenters: Design, Innovation, and Discovery (2nd ed.) , 2006 .

[12]  M. Walport,et al.  Looking for Landmarks: The Role of Expert Review and Bibliometric Analysis in Evaluating Scientific Publication Outputs , 2009, PloS one.

[13]  Vincent Larivière,et al.  The linguistic patterns and rhetorical structure of citation context: an approach using n-grams , 2016, Scientometrics.

[14]  Contrasting views on mexico's national system of researc hers , 2010 .

[15]  Stasa Milojevic,et al.  Quantifying the cognitive extent of science , 2015, J. Informetrics.

[16]  魏屹东,et al.  Scientometrics , 2018, Encyclopedia of Big Data.

[17]  Marek Gagolewski,et al.  Scientific impact assessment cannot be fair , 2013, J. Informetrics.

[18]  David Chavalarias,et al.  What’s wrong with Science? , 2016, Scientometrics.

[19]  Wolfgang Glänzel,et al.  What does scientometrics share with other "metrics" sciences? , 2013, J. Assoc. Inf. Sci. Technol..

[20]  Plergiorgio Strata,et al.  Citation analysis , 1995, Nature.

[21]  S. Rijcke,et al.  Bibliometrics: The Leiden Manifesto for research metrics , 2015, Nature.

[22]  Sidney Addelman,et al.  trans-Dimethanolbis(1,1,1-trifluoro-5,5-dimethylhexane-2,4-dionato)zinc(II) , 2008, Acta crystallographica. Section E, Structure reports online.

[23]  Manuel Núñez-Nickel,et al.  The multilayered nature of reference selection , 2009, J. Assoc. Inf. Sci. Technol..

[24]  N. Adler,et al.  When Knowledge Wins: Transcending the Sense and Nonsense of Academic Rankings , 2009 .

[25]  Martin Ricker.  A numerical algorithm with preference statements to evaluate the performance of scientists , 2014, Scientometrics.

[26]  Chaomei Chen,et al.  The Recurrence of Citations within a Scientific Article , 2015, ISSI.

[27]  Ying Ding,et al.  The distribution of references across texts: Some implications for citation analysis , 2013, J. Informetrics.

[28]  Cassidy R. Sugimoto,et al.  Introduction metrics & ASIS&T , 2012 .

[29]  Lutz Bornmann,et al.  OPEN PEN ACCESS CCESS , 2008 .

[30]  S. Rijcke,et al.  Bibliometrics: The Leiden Manifesto for research metrics. , 2015, Nature.

[31]  D. Aksnes,et al.  Peer reviews and bibliometric indicators: a comparative study at a Norwegian university , 2004 .

[32]  Michael H. MacRoberts,et al.  Problems of citation analysis: A study of uncited and seldom-cited influences , 2010, J. Assoc. Inf. Sci. Technol..

[33]  Lutz Bornmann,et al.  Does evaluative scientometrics lose its main focus on scientific quality by the new orientation towards societal impact? , 2016, Scientometrics.

[34]  Filippo Menczer,et al.  Quality versus quantity in scientific impact , 2014, J. Informetrics.

[35]  Linda Butler,et al.  Using a balanced approach to bibliometrics: quantitative performance measures in the Australian Research Quality Framework , 2008 .

[36]  Brigitte Tiefenthaler,et al.  Counting quality? The Czech performance-based research funding system , 2015 .