Measuring the scientific impact of e-research infrastructures: a citation based approach?

This micro-level study explores the extent that citation analysis provides an accurate and representative assessment of the use and impact of bioinformatics e-research infrastructure. The bioinformatic e-research infrastructure studied offers common tools used by life scientists to analyse and interpret genetic and protein sequence information. These e-resources therefore provide an interesting example with which to explore how representative citations are as acknowledgements of knowledge in the life sciences. The examples presented here suggest that there is a relation between number of visits to these databases and number of citations; however, a parallel finding shows how citation analysis frequently underestimates acknowledged use of the resources offered on this e-research infrastructure. The paper discusses the implications of the findings for various aspects of impact measurement and also considers how appropriate citation analysis is as a measurement of knowledge claims.

[1]  C. Hoogland,et al.  SWISS‐2DPAGE, ten years later , 2004, Proteomics.

[2]  R. Merton.  On the Shoulders of Giants: A Shandean Postscript , 1966 .

[3]  Norman M Kaplan,et al.  The norms of citation behavior: Prolegomena to the footnote , 1965 .

[4]  P. V. D. Besselaar,et al.  Evaluation of research in context - an approach and two cases , 2011 .

[5]  Isidro F. Aguillo,et al.  Measuring the usage of e-research infrastructure as an indicator of research activity , 2012, J. Assoc. Inf. Sci. Technol..

[6]  Bluma C. Peritz,et al.  Are methodological papers more cited than theoretical or empirical ones? The case of sociology , 1983, Scientometrics.

[7]  Henk F. Moed,et al.  Coverage and citation impact of oncological journals in the Web of Science and Scopus , 2008, J. Informetrics.

[8]  Péter Jacsó,et al.  Evaluation of citation enhanced scholarly databases , 2006 .

[9]  Thed N. van Leeuwen,et al.  New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications , 1995, Scientometrics.

[10]  Lutz Bornmann,et al.  What do citation counts measure? A review of studies on citing behavior , 2008, J. Documentation.

[11]  Amos Bairoch,et al.  The ENZYME database in 2000 , 2000, Nucleic Acids Res..

[12]  魏屹东,et al.  Scientometrics , 2018, Encyclopedia of Big Data.

[13]  K. A. McKibbon,et al.  How well are journal and clinical article characteristics associated with the journal impact factor? a retrospective cohort study. , 2012, Journal of the Medical Library Association : JMLA.

[14]  David King,et al.  Identifying Audiences of E-Infrastructures - Tools for Measuring Impact , 2012, PloS one.

[15]  Eugene Garfield,et al.  Random thoughts on citationology its theory and practice , 1998, Scientometrics.

[16]  Peter Weingart,et al.  Impact of bibliometrics upon the science system: Inadvertent consequences? , 2005, Scientometrics.

[17]  J. Senker.  Tacit Knowledge and Models of Innovation , 1995 .

[18]  G. Gilbert.  Referencing as Persuasion , 1977 .

[19]  S. Baldi.  Normative versus social constructivist processes in the allocation of citations : A network-analytic model , 1998 .

[20]  B. Martin,et al.  Assessing Basic Research : Some Partial Indicators of Scientific Progress in Radio Astronomy : Research Policy , 1987 .

[21]  O. H. Lowry,et al.  Protein measurement with the Folin phenol reagent. , 1951, The Journal of biological chemistry.

[22]  Alan Bridge,et al.  New and continuing developments at PROSITE , 2012, Nucleic Acids Res..

[23]  R D Appel,et al.  A new generation of information retrieval tools for biologists: the example of the ExPASy WWW server. , 1994, Trends in biochemical sciences.

[24]  Robert K. Merton,et al.  The Thomas Theorem and The Matthew Effect , 1995 .

[25]  Elisabeth Coudert,et al.  HAMAP: a database of completely sequenced microbial proteome sets and manually curated microbial protein families in UniProtKB/Swiss-Prot , 2008, Nucleic Acids Res..

[26]  S. Fingerman,et al.  Web of Science and Scopus: Current features and Capabilities. , 2006, Issues in Science and Technology Librarianship.

[27]  Zaida Chinchilla-Rodríguez,et al.  Coverage analysis of Scopus: A journal metric approach , 2007, Scientometrics.

[28]  Rafael Ball,et al.  Science indicators revisited - Science Citation Index versus SCOPUS: A bibliometric comparison of both citation databases , 2007, Inf. Serv. Use.

[29]  P.A.A. van den Besselaar,et al.  Are citations a complete measure for the impact of e-research infrastructures , 2013 .

[30]  Susan E. Cozzens,et al.  What do citations count? the rhetoric-first model , 1989, Scientometrics.