Three options for citation tracking: Google Scholar, Scopus and Web of Science

BackgroundResearchers turn to citation tracking to find the most influential articles for a particular topic and to see how often their own published papers are cited. For years researchers looking for this type of information had only one resource to consult: the Web of Science from Thomson Scientific. In 2004 two competitors emerged – Scopus from Elsevier and Google Scholar from Google. The research reported here uses citation analysis in an observational study examining these three databases; comparing citation counts for articles from two disciplines (oncology and condensed matter physics) and two years (1993 and 2003) to test the hypothesis that the different scholarly publication coverage provided by the three search tools will lead to different citation counts from each.MethodsEleven journal titles with varying impact factors were selected from each discipline (oncology and condensed matter physics) using the Journal Citation Reports (JCR). All articles published in the selected titles were retrieved for the years 1993 and 2003, and a stratified random sample of articles was chosen, resulting in four sets of articles. During the week of November 7–12, 2005, the citation counts for each research article were extracted from the three sources. The actual citing references for a subset of the articles published in 2003 were also gathered from each of the three sources.ResultsFor oncology 1993 Web of Science returned the highest average number of citations, 45.3. Scopus returned the highest average number of citations (8.9) for oncology 2003. Web of Science returned the highest number of citations for condensed matter physics 1993 and 2003 (22.5 and 3.9 respectively). The data showed a significant difference in the mean citation rates between all pairs of resources except between Google Scholar and Scopus for condensed matter physics 2003. For articles published in 2003 Google Scholar returned the largest amount of unique citing material for oncology and Web of Science returned the most for condensed matter physics.ConclusionThis study did not identify any one of these three resources as the answer to all citation tracking needs. Scopus showed strength in providing citing literature for current (2003) oncology articles, while Web of Science produced more citing material for 2003 and 1993 condensed matter physics, and 1993 oncology articles. All three tools returned some unique material. Our data indicate that the question of which tool provides the most complete set of citing literature may depend on the subject and publication year of a given article.

[1]  Alireza Noruzi.  Google Scholar: The New Generation of Citation Indexes , 2005 .

[2]  Andy R Weale,et al.  The level of non-citation of articles within a journal as a measure of quality: a comparison to the impact factor , 2004, BMC medical research methodology.

[3]  Dana L. Roth,et al.  The emergence of competitors to the Science Citation Index and the Web of Science , 2005 .

[4]  Kathryn J. Skhal,et al.  Wireless Information System for Emergency Responders (WISER) , 2006 .

[5]  E GARFIELD,et al.  Citation indexes for science; a new dimension in documentation through association of ideas. , 2006, Science.

[6]  Stephen S. Murray,et al.  The effect of use and access on citations , 2005, Inf. Process. Manag..

[7]  Gary Holden,et al.  Tracing Thought Through Time and Space , 2005, Social work in health care.

[8]  Nisa Bakkalbasi,et al.  An Examination of Citation Counts in a New Scholarly Communication Environment , 2005, D Lib Mag..

[9]  E. Garfield.  Citation indexes for science. A new dimension in documentation through association of ideas. 1955. , 1955, International journal of epidemiology.

[10]  Jim Henderson,et al.  Google Scholar: A source for clinicians? , 2005, Canadian Medical Association Journal.

[11]  Cecelia M. Brown.  The E-volution of preprints in the scholarly communication of physicists and astronomers , 2001, J. Assoc. Inf. Sci. Technol..

[12]  R. Greg,et al.  Notess. Scholarly Web Searching : Google Scholar and Scirus , 2005 .

[13]  Rob Kling,et al.  Not just a matter of time: Field differences and the shaping of electronic media in supporting scientific communication , 1999, J. Am. Soc. Inf. Sci..

[14]  Rob Kling,et al.  Scholarly Communication and the Continuum of Electronic Publishing , 1999, J. Am. Soc. Inf. Sci..

[15]  Cecelia M. Brown.  Information Seeking Behavior of Scientists in the Electronic Information Age: Astronomers, Chemists, Mathematicians, and Physicists , 1999, J. Am. Soc. Inf. Sci..

[16]  S. Bloch,et al.  Counting on citations: a flawed way to measure quality , 2003, The Medical journal of Australia.

[17]  Jonathan Furner,et al.  Scholarly communication and bibliometrics , 2005, Annu. Rev. Inf. Sci. Technol..

[18]  Eugene Garfield,et al.  Citation indexing - its theory and application in science, technology, and humanities , 1979 .

[19]  E. Garfield,et al.  Citation indexes for science. , 1956, Science.

[20]  Greg R. Notess,et al.  Locating uses for local search , 2005 .

[21]  Julianne Cheek,et al.  What's in a Number? Issues in Providing Evidence of Impact and Quality of Research(ers) , 2006, Qualitative health research.

[22]  Laura M. Felter,et al.  Google scholar, scirus, and the scholarly search revolution , 2005 .

[23]  Dangzhi Zhao,et al.  Challenges of scholarly publications on the Web to the evaluation of science - A comparison of author visibility on the Web and in print journals , 2005, Inf. Process. Manag..

[24]  P. Seglen,et al.  Education and debate , 1999, The Ethics of Public Health.

[25]  Irvin M Modlin,et al.  Information Assimilation and Distribution Challenges and Goals for Real and Virtual Journals , 2005, Journal of clinical gastroenterology.

[26]  Eugene Barsky,et al.  A look at Google Scholar, PubMed, and Scirus: comparisons and recommendations , 2005 .

[27]  E. Garfield.  The history and meaning of the journal impact factor. , 2006, JAMA.

[28]  Peter Ingwersen,et al.  Informetric analyses on the world wide web: methodological approaches to 'webometrics' , 1997, J. Documentation.

[29]  Johan Bollen,et al.  Toward alternative metrics of journal impact: A comparison of download and citation data , 2005, Inf. Process. Manag..

[30]  Andreas Thor,et al.  Citation analysis of database publications , 2005, SGMD.

[31]  D M D'Alessandro,et al.  Empowering children and families with information technology. , 2001, Archives of pediatrics & adolescent medicine.

[32]  Declan Butler,et al.  Science searches shift up a gear as Google starts Scholar engine , 2004, Nature.