A new look at evidence of scholarly citation in citation indexes and from web sources

A sample of 1,483 publications, representative of the scholarly production of LIS faculty, was searched in Web of Science (WoS), Google, and Google Scholar. The median number of citations found through WoS was zero for all types of publications except book chapters; the median for Google Scholar ranged from 1 for print/subscription journal articles to 3 for books and book chapters. For Google the median number of citations ranged from 9 for conference papers to 41 for books. A sample of the web citations was examined and classified as representing intellectual or non-intellectual impact. Almost 92% of the citations identified through Google Scholar represented intellectual impact — primarily citations from journal articles. Bibliographic services (non-intellectual impact) were the largest single contributor of citations identified through Google. Open access journal articles attracted more web citations but the citations to print/subscription journal articles more often represented intellectual impact. In spite of problems with Google Scholar, it has the potential to provide useful data for research evaluation, especially in a field where rapid and fine-grained analysis is desirable.

[1]  Nisa Bakkalbasi,et al.  Do Science Researchers Use Books , 2004 .

[2]  Steve Lawren.  Online or invisible ? , 2001 .

[3]  Thomas Schaffer,et al.  Psychology citations revisited: behavioral research in the age of electronic resources , 2004 .

[4]  Dana L. Roth,et al.  The emergence of competitors to the Science Citation Index and the Web of Science , 2005 .

[5]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007 .

[6]  R. Jackson,et al.  The Matthew Effect in Science , 1988, International journal of dermatology.

[7]  Nisa Bakkalbasi,et al.  An Examination of Citation Counts in a New Scholarly Communication Environment , 2005, D Lib Mag..

[8]  Dangzhi Zhao,et al.  Challenges of scholarly publications on the Web to the evaluation of science - A comparison of author visibility on the Web and in print journals , 2005, Inf. Process. Manag..

[9]  Lei Wang,et al.  Three options for citation tracking: Google Scholar, Scopus and Web of Science , 2006, Biomedical digital libraries.

[10]  Debora Shaw,et al.  Web citation data for impact assessment: A comparison of four science disciplines , 2005, J. Assoc. Inf. Sci. Technol..

[11]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007, J. Assoc. Inf. Sci. Technol..

[12]  S. Lawrence.  Free online availability substantially increases a paper's impact , 2001, Nature.

[13]  Alireza Noruzi.  Google Scholar: The New Generation of Citation Indexes , 2005 .

[14]  B. Cronin.  A hundred million acts of whimsy , 2005 .

[15]  Ronald Rousseau,et al.  Web-to-print citations and the humanities , 2006 .

[16]  Philip M. Davis,et al.  Do Open-Access articles really have a greater research impact? , 2006 .

[17]  Les Carr,et al.  The Open Research Web: A Preview of the Optimal and the Inevitable , 2006 .

[18]  Lokman I. Meho,et al.  Ranking the research productivity of library and information science faculty and schools: An evaluation of data sources and research methods , 2005, J. Assoc. Inf. Sci. Technol..

[19]  Robin B. Devin,et al.  The Serial/Monograph Ratio in Research Libraries: Budgeting in Light of Citation Studies , 1990 .

[20]  N. Jacobs,et al.  Open access : key strategic, technical and economic aspects , 2006 .

[21]  William G. Madow,et al.  The Literature of the Social Sciences. , 1960 .

[22]  Chris Neuhaus,et al.  The Depth and Breadth of Google Scholar: An Empirical Study , 2006 .

[23]  Péter Jacsó,et al.  Comparison and Analysis of the Citedness Scores in Web of Science and Google Scholar , 2005, ICADL.

[24]  Debora Shaw,et al.  Bibliographic and Web citations: What is the difference? , 2003, J. Assoc. Inf. Sci. Technol..

[25]  Richard K. Belew,et al.  Scientific impact quantity and quality: Analysis of two sources of bibliographic data , 2005, ArXiv.

[26]  K. Antelman.  Do Open-Access Articles Have a Greater Research Impact? , 2004 .

[27]  Daniel Pauly,et al.  Equivalence of results from two citation analyses: Thomson ISI's Citation Index and Google's Scholar service , 2005 .

[28]  Mike Thelwall,et al.  Motivations for URL citations to open access library and information science articles , 2006, Scientometrics.

[29]  P. Jacsó.  As we may search : Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases , 2005 .

[30]  M. Thelwall,et al.  Google Scholar citations and Google Web-URL citations: A multi-discipline exploratory analysis , 2007 .