Validation of Bibliometric Indicators in the Field of Microbiology: A Norwegian Case Study

This paper addresses two related issues regarding the validity of bibliometric indicators for the assessment of national performance within a particular scientific field. Firstly, the representativeness of a journal-based subject classification; and secondly, the completeness of the database coverage. Norwegian publishing in microbiology was chosen as a case, using the standard ISI-product National Science Indicators on Diskette (NSIOD) as a source database. By applying an "author-gated" retrieval procedure, we found that only 41 percent of all publications in NSIOD-indexed journals, expert-classified as microbiology, were included under the NSIOD-category Microbiology. Thus, the set of defining core journals only is clearly not sufficient to delineate this complex biomedical field. Furthermore, a subclassification of the articles into different subdisciplines of microbiology revealed systematic differences with respect to representation in NSIOD's Microbiology field; fish microbiology and medical microbiology are particularly underrepresented.In a second step, the individual publication lists from a sample of Norwegian microbiologists were collected and compared with the publications by the same authors, retrieved bibliometrically. The results showed that a large majority (94%) of the international scientific production in Norwegian microbiology was covered by the database NSIOD. Thus, insufficient subfield delineation, and not lack of coverage, appeared to be the main methodological problem in the bibliometric analysis of microbiology.

[1]  Diana Hicks,et al.  The difficulty of achieving full coverage of international social science literature and the bibliometric consequences , 1999, Scientometrics.

[2]  Paul Bourke,et al.  Publication types, citation rates and evaluation , 1996, Scientometrics.

[3]  Gabriel Pinski,et al.  Structure of the Biomedical Literature , 1976, J. Am. Soc. Inf. Sci..

[4]  W. G Zel.  THE NEED FOR STANDARDS IN BIBLIOMETRIC RESEARCH AND TECHNOLOGY , 1996 .

[5]  G. Folly,et al.  Some methodological problems in ranking scientists by citation analysis , 1981, Scientometrics.

[6]  Urs Schoepflin,et al.  Problems of representativity in the social sciences citation index , 1992 .

[7]  Wolfgang Glänzel,et al.  The need for standards in bibliometric research and technology , 2005, Scientometrics.

[8]  Henk F. Moed,et al.  Differences in the construction of sci based bibliometric indicators among various producers: A first over view , 1996, Scientometrics.

[9]  Rolf A. Zwaan,et al.  Assessing the usefulness of bibliometric indicators for the humanities and the social and beha vioural sciences: A comparative study , 1989, Scientometrics.

[10]  Jean-Christophe Doré,et al.  World science in 18 disciplinary areas: Comparative evaluation of the publication patterns of 48 countries over the period 1981–1992 , 1995, Scientometrics.

[11]  Anton J. Nederhof.  Delimitation of a medical research topic: interaction with experts in selecting a database and constructing a search strategy , 1991 .

[12]  H. Moed,et al.  The use of bibliometric data for the measurement of university research performance , 1985 .

[13]  María Bordons,et al.  Bibliometric analysis of publications of Spanish pharmacologists in the SCI (1984–89). Part II , 1992, Scientometrics.

[14]  Henk F. Moed,et al.  Measuring national output in physics: Delimitation problems , 1993, Scientometrics.

[15]  Tibor Braun,et al.  Cross-field normalization of scientometric indicators , 1996, Scientometrics.

[16]  Grant Lewison,et al.  The definition of biomedical research subfields with title keywords and application to the analysis of research outputs , 1996 .

[17]  Isabel Gómez,et al.  Coping with the problem of subject classification diversity , 2005, Scientometrics.

[18]  Henk F. Moed,et al.  Delimitation of scientific subfields using cognitive words from corporate addresses in scientific publications , 2005, Scientometrics.

[19]  Hariolf Grupp,et al.  The scientometric weight of 50 nations in 27 science areas, 1989–1993. Part I. All fields combined, mathematics, engineering, chemistry and physics , 2005, Scientometrics.

[20]  P. Seglen,et al.  Citations and journal impact factors: questionable indicators of research quality , 1997, Allergy.

[21]  Anton J. Nederhof,et al.  Development of bibliometric indicators for utility of research to users in society: Measurement of external knowledge transfer via publications in trade journals , 2005, Scientometrics.