The expansion of Google Scholar versus Web of Science: a longitudinal study

Web of Science (WoS) and Google Scholar (GS) are prominent citation services with distinct indexing mechanisms. Comprehensive knowledge about the growth patterns of these two citation services is lacking. We analyzed the development of citation counts in WoS and GS for two classic articles and 56 articles from diverse research fields, making a distinction between retroactive growth (i.e., the relative difference between citation counts up to mid-2005 measured in mid-2005 and citation counts up to mid-2005 measured in April 2013) and actual growth (i.e., the relative difference between citation counts up to mid-2005 measured in April 2013 and citation counts up to April 2013 measured in April 2013). One of the classic articles was used for a citation-by-citation analysis. Results showed that GS has substantially grown in a retroactive manner (median of 170 % across articles), especially for articles that initially had low citations counts in GS as compared to WoS. Retroactive growth of WoS was small, with a median of 2 % across articles. Actual growth percentages were moderately higher for GS than for WoS (medians of 54 vs. 41 %). The citation-by-citation analysis showed that the percentage of citations being unique in WoS was lower for more recent citations (6.8 % for citations from 1995 and later vs. 41 % for citations from before 1995), whereas the opposite was noted for GS (57 vs. 33 %). It is concluded that, since its inception, GS has shown substantial expansion, and that the majority of recent works indexed in WoS are now also retrievable via GS. A discussion is provided on quantity versus quality of citations, threats for WoS, weaknesses of GS, and implications for literature research and research evaluation.

[1]  Massimo Franceschet,et al.  A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar , 2010, Scientometrics.

[2]  Jeroen Bosman,et al.  Scopus reviewed and compared: the coverage and functionality of the citation database Scopus, including comparisons with Web of Science and Google Scholar , 2006 .

[3]  Nisa Bakkalbasi,et al.  An Examination of Citation Counts in a New Scholarly Communication Environment , 2005, D Lib Mag..

[4]  Jean Tague-Sutcliffe,et al.  An Introduction to Informetrics , 1992, Inf. Process. Manag..

[5]  Susanne Mikki,et al.  Comparing Google Scholar and ISI Web of Science for Earth Sciences , 2010, Scientometrics.

[6]  Sheila Wilson,et al.  Research Excellence Framework , 2013 .

[7]  P. Jacsó.  As we may search : Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases , 2005 .

[8]  M. M. Bradford.  A rapid and sensitive method for the quantitation of microgram quantities of protein utilizing the principle of protein-dye binding. , 1976, Analytical biochemistry.

[9]  Amartya Sen,et al.  On Some Debates in Capital Theory , 1974 .

[10]  J. Heremans,et al.  Immunochemical quantitation of antigens by single radial immunodiffusion. , 1965, Immunochemistry.

[11]  Philipp Mayr,et al.  An exploratory study of Google Scholar , 2007, Online Inf. Rev..

[12]  Kathryn J. Skhal,et al.  Wireless Information System for Emergency Responders (WISER) , 2006 .

[13]  Judit Bar-Ilan,et al.  Citations to the “Introduction to informetrics” indexed by WOS, Scopus and Google Scholar , 2010, Scientometrics.

[14]  Linda Butler,et al.  Extending citation analysis to non-source items , 2006, Scientometrics.

[15]  Jeffrey Beall,et al.  “Predatory” Open-Access Scholarly Publishers , 2010 .

[16]  Chris Neuhaus,et al.  The Depth and Breadth of Google Scholar: An Empirical Study , 2006 .

[17]  KoushaKayvan,et al.  Google Scholar citations and Google Web-URL citations: A multi-discipline exploratory analysis , 2007 .

[18]  Miguel A. García-Pérez,et al.  Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: A case study for the computation of h indices in Psychology , 2010, J. Assoc. Inf. Sci. Technol..

[19]  Joann M. Wleklinski,et al.  Studying google scholar: Wall to wall coverage? , 2005 .

[20]  Robert D. Simoni,et al.  The Most Highly Cited Paper in Publishing History: Protein Determination by Oliver H. Lowry , 2005 .

[21]  A. Einstein.  LENS-LIKE ACTION OF A STAR BY THE DEVIATION OF LIGHT IN THE GRAVITATIONAL FIELD. , 1936, Science.

[22]  魏屹东,et al.  Scientometrics , 2018, Encyclopedia of Big Data.

[23]  John Mingers,et al.  Counting the citations: a comparison of Web of Science and Google Scholar in the field of business and management , 2010, Scientometrics.

[24]  E. Garfield,et al.  The Most-Cited Papers of All Time, SCI 1945-1988. Part 1B. Superstars New to the SCI Top 100 , 1990 .

[25]  Péter Jacsó,et al.  Comparison and Analysis of the Citedness Scores in Web of Science and Google Scholar , 2005, ICADL.

[26]  Lei Wang,et al.  Three options for citation tracking: Google Scholar, Scopus and Web of Science , 2006, Biomedical digital libraries.

[27]  Péter Jacsó,et al.  Deflated, inflated and phantom citation counts , 2006, Online Inf. Rev..

[28]  Daniel Pauly,et al.  Equivalence of results from two citation analyses: Thomson ISI's Citation Index and Google's Scholar service , 2005 .

[29]  Thomas W. Conkling,et al.  Google Scholar’s Coverage of the Engineering Literature: An Empirical Study , 2008 .

[30]  Jeffrey Pomerantz.  Google Scholar and 100 Percent Availability of Information , 2006 .

[31]  Stéfan Jacques Darmoni,et al.  Is the coverage of google scholar enough to be used alone for systematic reviews , 2013, BMC Medical Informatics and Decision Making.

[32]  M. Thelwall,et al.  Google Scholar citations and Google Web-URL citations: A multi-discipline exploratory analysis , 2007 .

[33]  A. Kulkarni,et al.  Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. , 2009, JAMA.

[34]  E. Garfield.  The history and meaning of the journal impact factor. , 2006, JAMA.

[35]  S. D. De Groote,et al.  Coverage of Google Scholar, Scopus, and Web of Science: a case study of the h-index in nursing. , 2012, Nursing outlook.

[36]  U. K. Laemmli,et al.  Cleavage of Structural Proteins during the Assembly of the Head of Bacteriophage T4 , 1970, Nature.

[37]  Anne-Wil Harzing,et al.  A longitudinal study of Google Scholar coverage between 2012 and 2013 , 2013, Scientometrics.

[38]  Matthew E Falagas,et al.  Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses , 2007, FASEB journal : official publication of the Federation of American Societies for Experimental Biology.

[39]  D. B. Duncan.  MULTIPLE RANGE AND MULTIPLE F TESTS , 1955 .

[40]  Paulo Veríssimo,et al.  Handling self-citations using Google Scholar , 2009 .

[41]  Andreas Thor,et al.  Convergent validity of bibliometric Google Scholar data in the field of chemistry - Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar, Science Citation Index, Scopus, and Chemical Abstracts , 2009, J. Informetrics.

[42]  Xiaotian Chen.  Google Scholar's Dramatic Coverage Improvement Five Years after Debut 1 1 The author is grateful to Alice Chen of Duke University for editing this paper. , 2010 .

[43]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007, J. Assoc. Inf. Sci. Technol..

[44]  Mike Thelwall,et al.  Sources of Google Scholar citations outside the Science Citation Index: A comparison between four science disciplines , 2008, Scientometrics.

[45]  Nabil Amara,et al.  Counting citations in the field of business and management: why use Google Scholar rather than the Web of Science , 2012, Scientometrics.

[46]  Peder Olesen Larsen,et al.  The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index , 2010, Scientometrics.

[47]  Bela Gipp,et al.  Academic Search Engine Spam and Google Scholar's Resilience Against it , 2010 .

[48]  D. Mccormick.  Sequence the Human Genome , 1986, Bio/Technology.

[49]  Rachael Cathcart,et al.  Evaluating Google Scholar as a Tool for Information Literacy , 2005 .

[50]  James Testa.  The Thomson Reuters Journal Selection Process , 2009 .

[51]  Anne-Wil Harzing,et al.  A preliminary test of Google Scholar as a source for citation data: a longitudinal study of Nobel prize winners , 2013, Scientometrics.

[52]  Cyril Labbé.  Ike Antkare one of the great stars in the scientific firmament , 2010 .

[53]  Athina Tatsioni,et al.  Who is afraid of reviewers’ comments? Or, why anything can be published and anything can be cited , 2010, European journal of clinical investigation.

[54]  Christy Caldwell,et al.  Shifting Sands: Science Researchers on Google Scholar, Web of Science, and PubMed, with Implications for Library Collections Budgets. , 2010 .

[55]  Judit Bar-Ilan,et al.  Some measures for comparing citation databases , 2007, J. Informetrics.

[56]  Muhammad Akram,et al.  Text Book of Bioinformatics , 2011 .

[57]  Péter Jacsó,et al.  Google Scholar revisited , 2008, Online Inf. Rev..

[58]  Nicolás Robinson-García,et al.  Manipulating Google Scholar Citations and Google Scholar Metrics: simple, easy and tempting , 2012, ArXiv.

[59]  Michael Levine-Clark,et al.  A comparative analysis of social sciences citation tools , 2009, Online Inf. Rev..

[60]  Hans-Dieter Daniel,et al.  Data sources for performing citation analysis: an overview , 2008, J. Documentation.

[61]  A. Bandura.  Social Cognitive Theory of Mass Communication , 2001 .

[62]  松田 直人.  『Google Scholar』の利点 , 2009 .

[63]  E GARFIELD,et al.  Citation indexes for science; a new dimension in documentation through association of ideas. , 2006, Science.

[64]  Jeffrey Beall,et al.  Update: Predatory Open-Access Scholarly Publishers , 2010 .

[65]  R. Walser,et al.  Running with the Devil , 1993 .

[66]  O. H. Lowry,et al.  Protein measurement with the Folin phenol reagent. , 1951, The Journal of biological chemistry.

[67]  Ryoji Noyori,et al.  Asymmetric Catalysis by Chiral Metal Complexes , 1993 .