Google Scholar citations and Google Web/URL citations: A multi-discipline exploratory analysis

In this paper we introduce a new data gathering method “Web/URL Citation” and use it and Google Scholar as a basis to compare traditional and Web-based citation patterns across multiple disciplines. For this, we built a sample of 1,650 articles from 108 Open Access (OA) journals published in 2001 in four science and four social science disciplines. We recorded the number of citations to the sample articles using several methods based upon the ISI Web of Science, Google Scholar and the Google search engine (Web/URL citations). For each discipline, we found significant correlations between ISI citations and both Google Scholar and Google Web/URL citations; with similar results when using total or average citations, and when comparing within and across (most) journals. We also investigated disciplinary differences. Google Scholar citations were more numerous than ISI citations in our four social science disciplines as well as in computer science, suggesting that Google Scholar is a more comprehensive tool for citation tracking in the social sciences and perhaps also in fast-moving fields where conference papers are highly valued and published online. The results for Web/URL citations suggested that counting a maximum of one hit per site produces a better measure for assessing the impact of OA journals or articles, because replicated web citations are very common within individual sites. The results can be considered as additional evidence that there is some commonality between traditional and Web-extracted citations.

[1]  Jonathan Furner,et al.  Scholarly communication and bibliometrics , 2005, Annu. Rev. Inf. Sci. Technol..

[2]  Mike Thelwall,et al.  Scholarly Use of the Web: What Are the Key Inducers of Links to Journal Web Sites , 2003, J. Assoc. Inf. Sci. Technol..

[3]  Eun-Ja Shin.  Do Impact Factors change with a change of medium? A comparison of Impact Factors when publication is by paper and through parallel publishing , 2003, J. Inf. Sci..

[4]  Paul Nieuwenhuysen,et al.  Internet search engines - fluctuations in document accessibility , 2001, J. Documentation.

[5]  R. Rousseau.  Sitations: an exploratory study , 1997 .

[6]  Rob Kling,et al.  Electronic journals, the Internet, and scholarly communication , 2005, Annu. Rev. Inf. Sci. Technol..

[7]  Les Carr,et al.  The Access/Impact Problem and the Green and Gold Roads to Open Access: An Update , 2008 .

[8]  Stevan Harnad,et al.  The Future of Scholarly Skywriting , 1999 .

[9]  Mike Thelwall,et al.  Motivations for URL citations to open access library and information science articles , 2006, Scientometrics.

[10]  Debora Shaw,et al.  Bibliographic and Web citations: What is the difference? , 2003, J. Assoc. Inf. Sci. Technol..

[11]  S. Lawrence.  Free online availability substantially increases a paper's impact , 2001, Nature.

[12]  Peter Ingwersen,et al.  Perspective of webometrics , 2004, Scientometrics.

[13]  Mike Thelwall,et al.  Interpreting social science link analysis research: A theoretical framework , 2006, J. Assoc. Inf. Sci. Technol..

[14]  Jenny Fry,et al.  The Cultural Shaping of ICTs within Academic Fields: Corpus-based Linguistics as a Case Study , 2004, Lit. Linguistic Comput..

[15]  Stevan Harnad,et al.  Post-Gutenberg Galaxy: The Fourth Revolution in the Means of Production of Knowledge , 1991 .

[16]  Elisabeth Logan,et al.  Citation analysis using scientific publications on the Web as data source: A case study in the XML research area , 2002, Scientometrics.

[17]  Liwen Vaughan,et al.  New measurements for search engine evaluation proposed and tested , 2004, Inf. Process. Manag..

[18]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[19]  Judit Bar-Ilan,et al.  The use of web search engines in information science research , 2005, Annu. Rev. Inf. Sci. Technol..

[20]  Jenny Fry,et al.  The cultural shaping of scholarly communication: Explaining e-journal use within and across academic fields , 2004, ASIST.

[21]  Hak-Joon Kim,et al.  Motivations for hyperlinking in scholarly electronic articles: A qualitative study , 2000, J. Am. Soc. Inf. Sci..

[22]  Peter Ingwersen,et al.  The calculation of web impact factors , 1998, J. Documentation.

[23]  C. Lee Giles,et al.  Accessibility of information on the Web , 2000, INTL.

[24]  朱佳鸣.  Google Scholar Beta检索性能的初步分析 , 2005 .

[25]  R. Whitley.  The Intellectual and Social Organization of the Sciences (Second Edition: with new introductory chapter entitled 'Science Transformed? The Changing Nature of Knowledge Production at the End of the Twentieth Century') , 1984 .

[26]  Peter Ingwersen,et al.  Informetric analyses on the world wide web: methodological approaches to 'webometrics' , 1997, J. Documentation.

[27]  R. Greg,et al.  Notess. Scholarly Web Searching : Google Scholar and Scirus , 2005 .

[28]  Liwen Vaughan,et al.  Relationship between links to journal Web sites and impact factors , 2002, Aslib Proc..

[29]  Leo Egghe,et al.  New informetric aspects of the Internet: some reflections - many problems , 2000, J. Inf. Sci..

[30]  Judit Bar-Ilan.  Search engine results over time-a case study on search engine stability , 1998 .

[31]  Henk F. Moed,et al.  Citation Analysis in Research Evaluation , 1899 .

[32]  Liwen Vaughan,et al.  Webometrics , 2005, Annu. Rev. Inf. Sci. Technol..

[33]  Stevan Harnad,et al.  Scholarly Skywriting and the Prepublication Continuum of Scientific Inquiry , 1990 .

[34]  Maurice B. Line.  i in the Sky: Visions of the Information Future , 2001 .

[35]  K. Antelman.  Do Open-Access Articles Have a Greater Research Impact? , 2004 .

[36]  Alastair Smith,et al.  A Tale of Two Web Spaces: Comparing Sites Using Web Impact Factors. , 1999 .

[37]  R. Kling,et al.  Scholarly communication and the continuum of electronic publishing , 1999 .

[38]  S. Herring.  Use of Electronic Resources in Scholarly Electronic Journals: A Citation Analysis. , 2002 .

[39]  Nisa Bakkalbasi,et al.  An Examination of Citation Counts in a New Scholarly Communication Environment , 2005, D Lib Mag..

[40]  Alastair G. Smith.  Web links as research indicators: analogues of citations? , 2004, Inf. Res..

[41]  Stephen P. Harter.  The Impact of Electronic Journals on Scholarly Communication: A Citation Analysis , 1996 .

[42]  C. Lee Giles,et al.  Scholarly publishing in the Internet age: a citation analysis of computer science literature , 2001, Inf. Process. Manag..

[43]  Tim Brody,et al.  The effect of Open Access on Citation Impact , 2004 .

[44]  Debora Shaw,et al.  Web citation data for impact assessment: A comparison of four science disciplines , 2005, J. Assoc. Inf. Sci. Technol..