The Google scholar experiment: How to index false papers and manipulate bibliometric indicators

Google Scholar has been well received by the research community. Its promises of free, universal, and easy access to scientific literature coupled with the perception that it covers the social sciences and the humanities better than other traditional multidisciplinary databases have contributed to the quick expansion of Google Scholar Citations and Google Scholar Metrics: 2 new bibliometric products that offer citation data at the individual level and at journal level. In this article, we show the results of an experiment undertaken to analyze Google Scholar's capacity to detect citation‐counting manipulation. For this, we uploaded 6 documents to an institutional web domain that were authored by a fictitious researcher and referenced all the publications of the members of the EC3 research group at the University of Granada. The detection by Google Scholar of these papers caused an outburst in the number of citations included in the Google Scholar Citations profiles of the authors. We discuss the effects of such an outburst and how it could affect the future development of such products, at both the individual level and the journal level, especially if Google Scholar persists with its lack of transparency.

[1]  Ludo Waltman,et al.  The inconsistency of the h-index , 2011, J. Assoc. Inf. Sci. Technol..

[2]  José Luis Ortega,et al.  Science is all in the eye of the beholder: Keyword maps in Google scholar citations , 2012, J. Assoc. Inf. Sci. Technol..

[3]  S. Ceci,et al.  Peer-review practices of psychological journals: The fate of published articles, submitted again , 1982, Behavioral and Brain Sciences.

[4]  Christoph Bartneck,et al.  Detecting h-index manipulation through self-citation analysis , 2010, Scientometrics.

[5]  Bela Gipp,et al.  Academic Search Engine Spam and Google Scholar's Resilience Against it , 2010 .

[6]  Jöran Bela Erik Beel,et al.  Academic Search Engine Optimization (ASEO ): Optimizing Scholarly Literature for Google Scholar & Co. , 2010 .

[7]  Péter Jacsó,et al.  Google Scholar duped and deduped – the aura of “robometrics” , 2011 .

[8]  Rafael Ruiz-Pérez,et al.  Google Scholar como herramienta para la evaluación científica. , 2009 .

[9]  A. Swan,et al.  Researchers' use of academic libraries and their services: a report commissioned by the Research Information Network and the Consortium of Research Libraries , 2007 .

[10]  Mike Thelwall,et al.  Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus , 2011, J. Assoc. Inf. Sci. Technol..

[11]  José Luis Ortega,et al.  Institutional and country collaboration in an online service of scientific profiles: Google Scholar Citations , 2013, J. Informetrics.

[12]  Judit Bar-Ilan,et al.  Which h-index? — A comparison of WoS, Scopus and Google Scholar , 2008, Scientometrics.

[13]  Anne-Wil Harzing,et al.  Google Scholar as a new source for citation analysis , 2008 .

[14]  William H. Walters,et al.  Comparative Recall and Precision of Simple and Expert Searches in Google Scholar and Eight Other Databases , 2011 .

[15]  Ian Rowlands,et al.  Online use and information seeking behaviour: institutional and subject comparisons of UK researchers , 2009, J. Inf. Sci..

[16]  KoushaKayvan,et al.  Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus , 2011 .

[17]  Emilio Delgado López-Cózar,et al.  Google Scholar e índice h en biomedicina: la popularización de la evaluación bibliométrica , 2013 .

[18]  Mike Thelwall,et al.  How is science cited on the Web? A classification of google unique Web citations , 2007, J. Assoc. Inf. Sci. Technol..

[19]  D. Hicks.  The Four Literatures of Social Science , 2004 .

[20]  Nicolás Robinson-García,et al.  Manipulating Google Scholar Citations and Google Scholar Metrics: simple, easy and tempting , 2012, ArXiv.

[21]  Emilio Delgado López-Cózar,et al.  Repositories in Google Scholar Metrics or what is this document type doing in a place as such , 2012 .

[22]  Isidro F. Aguillo.  Is Google Scholar useful for bibliometrics? A webometric analysis , 2012, Scientometrics.

[23]  Emilio Delgado López-Cózar,et al.  Ranking journals: could Google Scholar Metrics be an alternative to Journal Citation Reports and Scimago Journal Rank? , 2013, Learn. Publ..

[24]  A. Sokal,et al.  Revelation: A Physicist Experiments with Cultural Studies , 2000 .

[25]  Matthew E. Falagas,et al.  The top-ten in journal impact factor manipulation , 2008, Archivum Immunologiae et Therapiae Experimentalis.

[26]  Rodrigo Costas,et al.  The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level , 2007, J. Informetrics.

[27]  Cyril Labbé.  Ike Antkare one of the great stars in the scientific firmament , 2010 .

[28]  Rodrigo Costas,et al.  Users, narcissism and control – tracking the impact of scholarly publications in the 21st century , 2012 .

[29]  Julie Carpenter,et al.  Researchers of Tomorrow: The research behaviour of Generation Y doctoral students , 2012, Inf. Serv. Use.

[30]  Erik Wilde,et al.  Academic Search Engine Optimization (ASEO) , 2010 .

[31]  Henk F. Moed,et al.  Handbook of Quantitative Science and Technology Research: The Use of Publication and Patent Statistics in Studies of S&T Systems , 2004 .

[32]  Emilio Delgado López-Cózar,et al.  Google Scholar Metrics: an unreliable tool for assessing scientific journals , 2012 .

[33]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007, J. Assoc. Inf. Sci. Technol..

[34]  Declan Butler,et al.  Computing giants launch free science metrics , 2011, Nature.

[35]  Margaret Markland,et al.  Institutional repositories in the UK: What can the Google user find there? , 2006, J. Libr. Inf. Sci..