Bibliometric methods: pitfalls and possibilities.

Bibliometric studies are increasingly being used for research assessment. Bibliometric indicators are strongly methodology-dependent but for all of them, various types of data normalization are an indispensable requirement. Bibliometric studies have many pitfalls; technical skill, critical sense and a precise knowledge about the examined scientific domain are required to carry out and interpret bibliometric investigations correctly.

[1]  Michel Zitt,et al.  Correcting glasses help fair comparisons in international science landscape: Country indicators as a function of ISI database delineation , 2003, Scientometrics.

[2]  M. Meyer.  Does science push technology? Patents citing scientific literature , 2000 .

[3]  Markus Gmür,et al.  Co-citation analysis and the search for invisible colleges: A methodological evaluation , 2004, Scientometrics.

[4]  Paul Bourke,et al.  Publication types, citation rates and evaluation , 1996, Scientometrics.

[5]  Ronald N. Kostoff,et al.  The use and misuse of citation analysis in research evaluation , 1998, Scientometrics.

[6]  R. Zetterström,et al.  Bibliometric data: a disaster for many non‐American biomedical journals , 2002, Acta paediatrica.

[7]  Henk F. Moed,et al.  Journal impact measures in bibliometric research , 2004, Scientometrics.

[8]  Francis Narin,et al.  Status report: Linkage between technology and science , 1992 .

[9]  Martha A. Harsanyi.  Multiple authors, multiple problems: bibliometrics and the study of scholarly collaboration: a literature review , 1993 .

[10]  Peter Vinkler,et al.  Subfield problems in applying the Garfield (Impact) Factors in practice , 2002, Scientometrics.

[11]  Leo Pavičić,et al.  Citation Context Versus the Frequency Counts of Citation Histories , 1998, J. Am. Soc. Inf. Sci..

[12]  Ruben Coronel,et al.  Impact factors: no totum pro parte by skewness of citation. , 2004, Cardiovascular research.

[13]  Thed N. van Leeuwen,et al.  Citation delay in interdisciplinary knowledge exchange , 2001, Scientometrics.

[14]  Isabel Gómez,et al.  Advantages and limitations in the use of impact factor measures for the assessment of research performance , 2002, Scientometrics.

[15]  Guido Van Hooydonk,et al.  Fractional Counting of Multiauthored Publications: Consequences for the Impact of Authors , 1997, J. Am. Soc. Inf. Sci..

[16]  M. Moravcsik,et al.  Some Results on the Function and Quality of Citations , 1975 .

[17]  H. Moed,et al.  The use of bibliometric data for the measurement of university research performance , 1985 .

[18]  J. Reedijk.  Sense and nonsense of science citation analyses: comments on the monopoly position of ISI and citation inaccuracies. Risks of possible misuse and biased citation and impact data. , 1998 .

[19]  Francis Narin,et al.  Bibliometric performance measures , 1996, Scientometrics.

[20]  A. Gupta,et al.  Pityriasis versicolor: quality of studies , 2004, The Journal of dermatological treatment.

[21]  Julie A. C. Virgo,et al.  A Statistical Procedure for Evaluating the Importance of Scientific Papers , 1977, The Library Quarterly.

[22]  Martin Meyer,et al.  What is Special about Patent Citations? Differences between Scientific and Patent Citations , 2000, Scientometrics.

[23]  Guido Van Hooydonk,et al.  Standardizing Relative Impacts: Estimating the Quality of Research from Citation Counts , 1998, J. Am. Soc. Inf. Sci..

[24]  Richard C. Anderson,et al.  Comparison of peer and citation assessment of the influence of scientific journals , 1980, J. Am. Soc. Inf. Sci..

[25]  Ed J. Rinia,et al.  COMPARATIVE ANALYSIS OF A SET OF BIBLIOMETRIC INDICATORS AND CENTRAL PEER REVIEW CRITERIA. EVALUATION OF CONDENSED MATTER PHYSICS IN THE NETHERLANDS , 1998 .

[26]  Robert N. Broadus.  An investigation of the validity of bibliographic citations , 1983, J. Am. Soc. Inf. Sci..

[27]  Héctor Guerrero,et al.  A robust formula to credit authors for their publications , 2004, Scientometrics.

[28]  F S Chew,et al.  How research becomes knowledge in radiology: an analysis of citations to published papers. , 1988, AJR. American journal of roentgenology.

[29]  Young Ha Park,et al.  Researcher contributions and fulfillment of ICMJE authorship criteria: analysis of author contribution lists in research articles with multiple authors published in radiology. International Committee of Medical Journal Editors. , 2003, Radiology.

[30]  Michael J. Moravcsik,et al.  Variation of the nature of citation measures with journals and scientific specialties , 1978, J. Am. Soc. Inf. Sci..

[31]  H. Small.  A Co-Citation Model of a Scientific Specialty: A Longitudinal Study of Collagen Research , 1977 .

[32]  Charles Oppenheim,et al.  The Correlation between citation counts and the 1992 Research Assessment Exercise Ratings for British Library and Information Science University departments , 1995, J. Documentation.

[33]  D. Cases,et al.  How can we investigate citation behavior?: a study of reasons for citing literature in communication , 2000 .

[34]  Concepción S. Wilson,et al.  Measuring the citation impact of research journals in clinical neurology: A structural equation modelling analysis , 2004, Scientometrics.

[35]  A. F. J. VAN RAAN,et al.  In matters of quantitative studies of science the fault of theorists is offering too little and asking too much , 1998, Scientometrics.

[36]  F. Narin,et al.  Bibliometrics/Theory, Practice and Problems , 1994 .

[37]  T. Opthof,et al.  Sense and nonsense about the impact factor. , 1997, Cardiovascular research.

[38]  D. Hicks.  Limitations of Co-Citation Analysis as a Tool for Science Policy , 1987 .

[39]  M. M. Kessler.  Bibliographic coupling between scientific papers , 1963 .

[40]  M. Porta,et al.  The bibliographic "impact factor" of the Institute for Scientific Information: how relevant is it really for public health journals? , 1996, Journal of epidemiology and community health.

[41]  Derek de Solla Price,et al.  A general theory of bibliometric and other cumulative advantage processes , 1976, J. Am. Soc. Inf. Sci..

[42]  D. King.  The scientific impact of nations , 2004, Nature.

[43]  Howard D. White,et al.  Authors as citers over time , 2001, J. Assoc. Inf. Sci. Technol..

[44]  Henk F. Moed,et al.  Possible inaccuracies occurring in citation analysis , 1989, J. Inf. Sci..

[45]  Henk F. Moed,et al.  Improving the accuracy of Institute for Scientific Information's journal impact factors , 1995 .

[46]  Tibor Braun,et al.  Reference standards for citation based assessments , 2005, Scientometrics.

[47]  Dag W. Aksnes,et al.  A macro study of self-citation , 2003, Scientometrics.

[48]  F D Wolinsky,et al.  Rating journals in health care administration. The use of bibliometric measures. , 1993, Medical care.

[49]  B. New,et al.  The Case Against , 2018, Journal of Pharmacy Practice and Research.

[50]  Robert F. Bornstein,et al.  The predictive validity of peer review: A neglected issue , 1991, Behavioral and Brain Sciences.

[51]  Anthony F. J. van Raan,et al.  The influence of international collaboration on the impact of research results , 1998, Scientometrics.

[52]  Wolfgang Glänzel,et al.  A bibliometric study on ageing and reception processes of scientific literature , 1995, J. Inf. Sci..

[53]  S. Gisvold,et al.  Citation analysis and journal impact factors – is the tail wagging the dog? , 1999, Acta anaesthesiologica Scandinavica.

[54]  T. Brooks.  Evidence of complex citer motivations , 1986, J. Am. Soc. Inf. Sci..

[55]  Radosvet Todorov.  Displaying content of scientific journals: A co-heading analysis , 2005, Scientometrics.

[56]  Vwani P. Roychowdhury,et al.  Stochastic modeling of citation slips , 2004, Scientometrics.

[57]  Dag W. Aksnes,et al.  The effect of highly cited papers on national citation indicators , 2004, Scientometrics.

[58]  Anthony F. J. van Raan,et al.  Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises , 1996, Scientometrics.

[59]  M. White,et al.  A Qualitative Study of Citing Behavior: Contributions, Criteria, and Metalevel Documentation Concerns , 1997, The Library Quarterly.

[60]  Henk F. Moed,et al.  The publication output and impact of academic chemistry research in the Netherlands during the 1980s: bibliometric analyses and policy implications , 1996 .

[61]  A. Raan,et al.  A bibliometric analysis of six economics research groups: A comparison with peer review , 1993 .

[62]  R. Tijssen.  Global and domestic utilization of industrial relevant science: patent citation analysis of science-technology interactions and knowledge flows , 2001 .

[63]  Ronald N. Kostoff,et al.  Citation analysis of research performer quality , 2004, Scientometrics.

[64]  Concepción S. Wilson,et al.  Informetric studies using databases: Opportunities and challenges , 2003, Scientometrics.

[65]  Rolf A. Zwaan,et al.  Assessing the usefulness of bibliometric indicators for the humanities and the social and beha vioural sciences: A comparative study , 1989, Scientometrics.

[66]  A A Sandberg,et al.  The journal "impact factor": a misnamed, misleading, misused measure. , 1998, Cancer genetics and cytogenetics.

[67]  Andrea Scharnhorst,et al.  Competition in science and the Matthew core journals , 2001, Scientometrics.

[68]  Wayne Hall.  Highs and lows , 1997, The Lancet.

[69]  Henk F. Moed,et al.  The application of bibliometric indicators: Important field- and time-dependent factors to be considered , 1985, Scientometrics.

[70]  Johan Albert Wallin.  Forskningen ved Det Sundhedsvidenskabelige Fakultet Syddansk Universitet 1998-2000: En bibliometrisk undersøgelse , 2004 .

[71]  M. H. MacRoberts,et al.  The Negational Reference: or the Art of Dissembling , 1984 .

[72]  Howard D. White,et al.  Reward, persuasion, and the Sokal Hoax: A study in citation identities , 2004, Scientometrics.

[73]  Thed N. van Leeuwen,et al.  Development and application of journal impact measures in the Dutch science system , 2002, Scientometrics.

[74]  Wolfgang Glänzel,et al.  The need for standards in bibliometric research and technology , 2005, Scientometrics.

[75]  E. Garfield.  From Citation Indexes to Informetrics: Is the Tail Now Wagging the Dog ? , 1998 .

[76]  Helmut A. Abt,et al.  Do Important Papers Produce High Citation Counts? , 2000, Scientometrics.

[77]  P Billesbølle,et al.  [The frequency of citations in Ugeskrift for Laeger--which journals were cited in 1986?]. , 1988, Ugeskrift for laeger.

[78]  Wolfgang Glänzel,et al.  A bibliometric approach to the role of author self-citations in scientific communication , 2004, Scientometrics.

[79]  Radosvet Todorov,et al.  Mapping Australian geophysics: A co-heading analysis , 1990, Scientometrics.

[80]  Julien Taieb,et al.  Truth Survival in Clinical Research: An Evidence-Based Requiem? , 2002, Annals of Internal Medicine.

[81]  Tibor Braun,et al.  Cross-field normalization of scientometric indicators , 1996, Scientometrics.

[82]  Charles Oppenheim,et al.  The correlation between citation counts and the 1992 research assessment exercise ratings for British research in genetics, anatomy and archaeology , 1997, J. Documentation.

[83]  Francis Narin,et al.  Technological performance assessments based on patents and patent citations , 1984, IEEE Transactions on Engineering Management.

[84]  Per Ottar Seglen,et al.  The skewness of science , 1992 .

[85]  Jacques Michel,et al.  Patent citation analysis.A closer look at the basic input data from patent search reports , 2001, Scientometrics.

[86]  Richard E. Stern,et al.  Uncitedness in the biomedical literature , 1990, J. Am. Soc. Inf. Sci..

[87]  Terrence A. Brooks,et al.  Private acts and public objects: An investigation of citer motivations , 1985, J. Am. Soc. Inf. Sci..

[88]  A. Raan.  Measuring Science: Capita Selecta of Current Main Issues , 2004 .

[89]  S. Baldi.  Normative versus social constructivist processes in the allocation of citations : A network-analytic model , 1998 .

[90]  Blaise Cronin,et al.  Semiotics and evaluative bibliometrics , 2000, J. Documentation.

[91]  C. Platell,et al.  Half-life of truth in surgical literature , 1997, The Lancet.

[92]  Loet Leydesdorff,et al.  Theories of Citation , 1998 .

[93]  R. Merton.  The Matthew Effect in Science , 1968, Science.

[94]  M. V. Simkin,et al.  Copied citations create renowned papers , 2003, cond-mat/0305150.

[95]  V. Cano,et al.  Citation behavior: Classification, utility, and location , 1989, JASIS.

[96]  Jeppe Nicolaisen,et al.  The J-shaped distribution of citedness , 2002, J. Documentation.

[97]  Meenen Nm.  The impact factor--a reliable sciento-metric parameter? , 1996 .

[98]  Kim Kapseon.  The motivation for citing specific references by social scientists in Korea: The phenomenon of co-existing references , 2004 .

[99]  Ming-Yueh Tsay,et al.  The nature and relationship between the productivity of journals and their citations in semiconductor literature , 2004, Scientometrics.

[100]  Lydia L. Lange,et al.  Citation Counts of Multi-Authored Papers — First-named Authors and Further Authors , 2001, Scientometrics.

[101]  Henry G. Small,et al.  Co-citation in the scientific literature: A new measure of the relationship between two documents , 1973, J. Am. Soc. Inf. Sci..

[102]  Alan E. Bayer,et al.  Validity of citation criteria for assessing the influence of scientific publications: New evidence with peer assessment , 1983, J. Am. Soc. Inf. Sci..

[103]  T. V. Leeuwen,et al.  Assessment of the scientific basis of interdisciplinary, applied research: Application of bibliometric methods in Nutrition and Food Research , 2002 .

[104]  Ronald N. Kostoff,et al.  Citation mining: Integrating text mining and bibliometrics for research user profiling , 2001, J. Assoc. Inf. Sci. Technol..

[105]  Michael H. MacRoberts,et al.  Problems of citation analysis , 1996, Scientometrics.

[106]  Mike Clarke,et al.  Citation of group-authored papers , 2002, The Lancet.

[107]  Henk F. Moed,et al.  Assessing the quality of scholarly journals in Linguistics:An alternative to citation-based journal impact factors , 2001, Scientometrics.

[108]  Helmut A. Abt.  The Reference-Frequency Relation in the Physical Sciences , 2004, Scientometrics.

[109]  P. Lawrence.  The politics of publication , 2003, Nature.

[110]  Thomas E. Nisonger,et al.  The relationship between international editorial board composition and citation measures in political science, business, and genetics journals , 2002, Scientometrics.

[111]  Ronald N. Kostoff,et al.  Phenomenological Approach to Profile Impact of Scientific Research: citation Mining , 2002, Adv. Complex Syst..

[112]  Wolfgang Glänzel,et al.  A Bibliometric Study of Reference Literature in the Sciences and Social Sciences , 1999, Inf. Process. Manag..

[113]  P. Seglen,et al.  Education and debate , 1999, The Ethics of Public Health.

[114]  Helmut A. Abt.  WHAT FRACTION OF LITERATURE REFERENCES ARE INCORRECT , 1992 .

[115]  Debora Shaw,et al.  Identity-creators and image-makers: Using citation analysis and thick description to put authors in their place , 2002, Scientometrics.

[116]  Rolf A. Zwaan,et al.  Quality Judgments of Journals as Indicators of Research Performance in the Humanities and the Social and Behavioral Sciences. , 1991 .

[117]  Narongrit Sombatsompop,et al.  A modified method for calculating the Impact Factors of journals in ISI Journal Citation Reports: Polymer Science Category in 1997–2001 , 2004, Scientometrics.

[118]  G. Gilbert.  Referencing as Persuasion , 1977 .

[119]  Subir K Sen,et al.  A mathematical extension of the idea of bibliographic coupling and its applications , 1983 .

[120]  Per Ottar Seglen,et al.  Causal relationship between article citedness and journal impact , 1994 .

[121]  Thed N. van Leeuwen,et al.  The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence , 2003, Scientometrics.

[122]  Kim J. Vicente,et al.  Can scientific impact be judged prospectively? A bibliometric test of Simonton"s model of creative productivity , 2004, Scientometrics.

[123]  Henry G. Small,et al.  On the shoulders of Robert Merton: Towards a normative theory of citation , 2004, Scientometrics.

[124]  Diane H. Sonnenwald,et al.  Citation ranking versus peer evaluation of senior faculty research performance: a case study of Kurdish scholarship , 2000 .

[125]  Wolfgang Glänzel,et al.  Towards a model for diachronous and synchronous citation analyses , 2004, Scientometrics.

[126]  Michael H. MacRoberts,et al.  Quantitative Measures of Communication in Science: A Study of the Formal Level , 1986 .

[127]  Blaise Cronin,et al.  Comparative citation rankings of authors in monographic and journal literature: a study of sociology , 1997, J. Documentation.

[128]  Robert West,et al.  What do citation counts count for in the field of addiction? An empirical evaluation of citation counts and their link with peer ratings of quality. , 2002, Addiction.

[129]  S. M. Lawani,et al.  Some bibliometric correlates of quality in scientific research , 2005, Scientometrics.

[130]  N. M. Meenen.  Der Impact-Faktor — ein zuverlässiger scientometrischer Parameter? , 2007, Unfallchirurgie.

[131]  Guido Van Hooydonk,et al.  Methods for accrediting publications to authors or countries: consequences for evaluation studies , 2000 .

[132]  Guido Van Hooydonk,et al.  Journal production and journal impact factors , 1996 .

[133]  Wolfgang Glänzel,et al.  A new methodological approach to bibliographic coupling and its application to the national, regional and institutional level , 2005, Scientometrics.

[134]  Peter Vinkler.  Possible causes of differences in information impact of journals from different subfields , 2005, Scientometrics.

[135]  Wolfgang Glänzel,et al.  The influence of author self-citations on bibliometric macro indicators , 2004, Scientometrics.

[136]  Henk F. Moed,et al.  In basic science the percentage of “authoritative” references decreases as bibliographies become shorter , 2004, Scientometrics.

[137]  Endre Szva-Kovts,et al.  Indirect-collective referencing (ICR) in the elite journal literature of physics. I. A literature science study on the journal level , 2001 .

[138]  Grant Lewison.  Researchers" and users" perceptions of the relative standing of biomedical papers in different journals , 2004, Scientometrics.

[139]  Kimberly S. Hamilton,et al.  The increasing linkage between U.S. technology and public science , 1997 .

[140]  Thed N. van Leeuwen,et al.  Critical comments on Institute for Scientific Information impact factors: a sample of inorganic molecular chemistry journals , 1999, J. Inf. Sci..

[141]  Endre Száva-Kováts.  Indirect-collective referencing (ICR) in the elite journal literature of physics. I. a literature science study on the journal level , 2001 .