Using Google Scholar for journal impact factors and the h‐index in nationwide publishing assessments in academia – siren songs and air‐raid sirens

Purpose – Google Scholar has been increasingly used in the past six to seven years as a highly efficient information source and service by librarians and other information professionals. The problem is when Google Scholar is touted and used as a bibliometric/scientometric tool and resource in the assessment of the quantity (productivity) and quality (impact) of research publications, in formal and informal ways, for decisions related to tenure, promotion and grant applications of individual researchers and research groups, as well as in journal subscriptions and cancellations. This paper aims to examine this issue.Design/methodology/approach – The paper discusses the use of Google Scholar for journal impact factors and the h‐index in nationwide publishing assessments in academia. It focuses on the issues of access and excess in Google Scholar: the innate limits of Google Scholar and those imposed by its developers on the users.Findings – The paper reveals that issues of access and excess in Google Scholar...

[1]  Péter Jacsó,et al.  The pros and cons of computing the h-index using Google Scholar , 2008, Online Inf. Rev..

[2]  Ronald Rousseau,et al.  Median and percentile impact factors: A set of new indicators , 2005, Scientometrics.

[3]  Wolfgang Glänzel.  The multi-dimensionality of journal impact , 2008, Scientometrics.

[4]  Judit Bar-Ilan,et al.  Which h-index? — A comparison of WoS, Scopus and Google Scholar , 2008, Scientometrics.

[5]  Gary E. Gorman.  "They can't read, but they sure can count": Flawed rules of the journal rankings game , 2008, Online Inf. Rev..

[6]  Péter Jacsó.  Grim tales about the impact factor and the h-index in the Web of Science and the Journal Citation Reports databases: reflections on Vanclay’s criticism , 2012, Scientometrics.

[7]  Péter Jacsó,et al.  Google Scholar's Ghost Authors. , 2009 .

[8]  Péter Jacsó.  How big is a database versus how is a database big , 2007, Online Inf. Rev..

[9]  Stephen J. Bensman.  The impact factor: its place in Garfield’s thought, in science evaluation, and in library collection management , 2011, Scientometrics.

[10]  Juan Miguel Campanario.  Empirical study of journal impact factors obtained using the classical two-year citation window versus a five-year citation window , 2010, Scientometrics.

[11]  Péter Jacsó,et al.  Deflated, inflated and phantom citation counts , 2006, Online Inf. Rev..

[12]  Péter Jacsó.  The dimensions of cited reference enhanced database subsets , 2007, Online Inf. Rev..

[13]  Jerome K. Vanclay,et al.  An evaluation of the Australian Research Council's journal ranking , 2010, J. Informetrics.

[14]  Henk F. Moed,et al.  Improving the accuracy of Institute for Scientific Information's journal impact factors , 1995 .

[15]  Jerome K. Vanclay,et al.  Gauging the impact of journals , 2008 .

[16]  Péter Jacsó,et al.  The plausibility of computing the h-index of scholarly productivity and impact using reference-enhanced databases , 2008, Online Inf. Rev..

[17]  E. Garfield.  Fortnightly Review: How can impact factors be improved? , 1996 .

[18]  Wolfgang Glänzel,et al.  Statistical reliability of comparisons based on the citation impact of scientific publications , 2005, Scientometrics.

[19]  Wolfgang Glänzel,et al.  A Hirsch-type index for journals , 2006, Scientometrics.

[20]  Stevan Harnad,et al.  Validating research performance metrics against peer rankings , 2008 .

[21]  T. E. Nisonger.  A methodological issue concerning the use of Social sciences citation index journal citation reports impact factor data for journal ranking , 1994 .

[22]  Péter Jacsó,et al.  Google Scholar revisited , 2008, Online Inf. Rev..

[23]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007 .

[24]  Péter Jacsó,et al.  Testing the Calculation of a Realistic h-index in Google Scholar, Scopus, and Web of Science for F. W. Lancaster , 2008, Libr. Trends.

[25]  Eugene Garfield,et al.  New factors in the evaluation of scientific literature through citation indexing , 1963 .

[26]  Péter Jacsó,et al.  Google Scholar Author Citation Tracker: is it too little, too late? , 2012 .

[27]  Alexander I. Pudovkin,et al.  Rank normalization of impact factors will resolve Vanclay’s dilemma with TRIF , 2012, Scientometrics.

[28]  A. Ben Wagner.  Percentile-Based Journal Impact Factors: A Neglected Collection Development Metric. , 2009 .

[29]  Barbara Quint,et al.  Changes at Google Scholar: A Conversation with Anurag Acharya , 2008 .

[30]  Peng Dong,et al.  The "impact factor" revisited , 2005, Biomedical digital libraries.

[31]  Lutz Bornmann,et al.  OPEN PEN ACCESS CCESS , 2008 .

[32]  Charles Oppenheim.  Do Citations Count? Citation Indexing and the Research Assessment Exercise (RAE) , 1996 .

[33]  Jerome K. Vanclay,et al.  Impact factor: outdated artefact or stepping-stone to journal certification? , 2011, Scientometrics.

[34]  Ronald Rousseau,et al.  Journal Evaluation: Technical and Practical Issues, , 2002, Libr. Trends.

[35]  P. Seglen,et al.  Education and debate , 1999, The Ethics of Public Health.

[36]  Jerome K. Vanclay,et al.  Ranking forestry journals using the h-index , 2007, J. Informetrics.

[37]  Péter Jacsó,et al.  Metadata mega mess in Google Scholar , 2010, Online Inf. Rev..

[38]  Thomas E. Nisonger,et al.  The Benefits and Drawbacks of Impact Factor for Journal Collection Management in Libraries , 2004 .

[39]  Péter Jacsó,et al.  Calculating the h-index and other bibliometric and scientometric indicators from Google Scholar with the Publish or Perish software , 2009, Online Inf. Rev..

[40]  Péter Jacsó,et al.  Content Evaluation of Databases. , 1997 .

[41]  E. Garfield.  Journal impact factor: a brief review. , 1999, CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne.

[42]  M. Freitas,et al.  The Impact Factor , 2013 .

[43]  Linda Butler.  The devil is in the detail: Concerns about Vanclay's analysis of Australian journal rankings , 2011, J. Informetrics.

[44]  E. Garfield.  Citation analysis as a tool in journal evaluation. , 1972, Science.