The counting house: measuring those who count. Presence of Bibliometrics, Scientometrics, Informetrics, Webometrics and Altmetrics in the Google Scholar Citations, ResearcherID, ResearchGate, Mendeley & Twitter

Following in the footsteps of the model of scientific communication, which has recently gone through a metamorphosis (from the Gutenberg galaxy to the Web galaxy), a change in the model and methods of scientific evaluation is also taking place. A set of new scientific tools are now providing a variety of indicators which measure all actions and interactions among scientists in the digital space, making new aspects of scientific communication emerge. In this work we present a method for ―capturing‖ the structure of an entire scientific community (the Bibliometrics, Scientometrics, Informetrics, Webometrics, and Altmetrics community) and the main agents that are part of it (scientists, documents, and sources) through the lens of Google Scholar Citations (GSC). Additionally, we compare these author ―portraits‖ to the ones offered by other profile or social platforms currently used by academics (ResearcherID, ResearchGate, Mendeley, and Twitter), in order to test their degree of use, completeness, reliability, and the validity of the information they provide. A sample of 814 authors (researchers in Bibliometrics with a public profile created in GSC) was subsequently searched in the other platforms, collecting the main indicators computed by each of them. The data collection was carried out on September, 2015. The Spearman correlation (α= 0.05) was applied to these indicators (a total of 31), and a Principal Component Analysis was carried out in order to reveal the relationships among metrics and platforms as well as the possible existence of metric clusters. We found that it is feasible to depict an accurate representation of the current state of the Bibliometrics community using data from GSC (the most influential authors, documents, journals, and publishers). Regarding the number of authors found in each platform, GSC takes the first place (814 authors), followed at a distance by ResearchGate (543), which is currently growing at a vertiginous speed. The number of Mendeley profiles is high, although 17.1% of them are basically empty. ResearcherID is also affected by this issue (34.45% of the profiles are empty), as is Twitter (47% of the Twitter accounts have published less than 100 tweets). Only 11% of our sample (93 authors) have created a profile in all the platforms analyzed in this study. From the PCA, we found two kinds of impact on the Web: first, all metrics related to academic impact. This first group can further be divided into usage metrics (views and downloads) and citation metrics. Second, all metrics related to connectivity and popularity (followers). ResearchGate indicators, as well as Mendeley readers, present a high correlation to all the indicators from GSC, but only a moderate correlation to the indicators in ResearcherID. Twitter indicators achieve only low correlations to the rest of the indicators, the highest of these being to GSC (0.42-0.46), and to Mendeley (0.41-0.46). Lastly, we present a taxonomy of all the errors that may affect the reliability of the data contained in each of these platforms, with a special emphasis in GSC, since it has been our main source of data. These errors alert us to the danger of blindly using any of these platforms for the assessment of individuals, without verifying the veracity and exhaustiveness of the data. In addition to this working paper, we also have made available a website where all the data obtained for each author and the results of the analysis of the most cited documents can be found: Scholar Mirrors.

[1]  Wolfgang Glänzel,et al.  Little scientometrics, big scientometrics ... and beyond? , 1994, Scientometrics.

[2]  R. Whitley.  The Intellectual and Social Organization of the Sciences (Second Edition: with new introductory chapter entitled 'Science Transformed? The Changing Nature of Knowledge Production at the End of the Twentieth Century') , 1984 .

[3]  Jonathan Furner,et al.  Scholarly communication and bibliometrics , 2005, Annu. Rev. Inf. Sci. Technol..

[4]  Enrique Orduña-Malea,et al.  Does Google Scholar contain all highly cited documents (1950-2013)? , 2014, ArXiv.

[5]  Arthur W. Elias,et al.  Control and Elimination of Errors in ISI Services. , 1966 .

[6]  Vincent Larivière,et al.  A bibliometric chronicling of library and information science's first hundred years , 2012, J. Assoc. Inf. Sci. Technol..

[7]  Tony Becher,et al.  Academic Tribes and Territories: Intellectual Enquiry and the Cultures of Disciplines , 2001 .

[8]  Salvador Gorbea Portal.  Principios teóricos y metodológicos de los estudios métricos de la información , 1994 .

[9]  Anthony F. J. van Raan,et al.  Scientometrics: State-of-the-art , 2006, Scientometrics.

[10]  Tove Faber Frandsen,et al.  Bibliometric evolution: Is the journal of the association for information science and technology transforming into a specialty Journal? , 2015, J. Assoc. Inf. Sci. Technol..

[11]  Péter Jacsó,et al.  Metadata mega mess in Google Scholar , 2010, Online Inf. Rev..

[12]  Péter Jacsó,et al.  Deflated, inflated and phantom citation counts , 2006, Online Inf. Rev..

[13]  Kramer Bianca,et al.  101 Innovations in Scholarly Communication - the Changing Research Workflow , 2015 .

[14]  Péter Jacsó,et al.  Google Scholar revisited , 2008, Online Inf. Rev..

[15]  Lluís Codina,et al.  Presencia de las universidades españolas en las redes sociales digitales científicas: caso de los estudios de Comunicación , 2015 .

[16]  J. Nicolaisen.  Bibliometrics and Citation Analysis: From the Science Citation Index to Cybermetrics , 2010 .

[17]  Tony Becher,et al.  Academic Tribes and Territories Intellectual enquiry and the culture of disciplines SECOND EDITION , 2001 .

[18]  B. C. Brookes.  Comments on the scope of bibliometrics , 1988 .

[19]  M. Castells.  La Galaxia Internet , 2001 .

[20]  Enrique Orduña-Malea,et al.  Methods for estimating the size of Google Scholar , 2014, Scientometrics.

[21]  Katherine W. McCain,et al.  Visualizing a discipline: an author co-citation analysis of information science, 1972–1995 , 1998 .

[22]  Poyer Rk,et al.  Inaccurate references in significant journals of science. , 1979 .

[23]  Isidro F. Aguillo,et al.  Can a personal website be useful as an information source to assess individual scientists? The case of European highly cited researchers , 2013, Scientometrics.

[24]  Eugene Garfield.  Journal Editors Awaken to the Impact of Citation Errors. How We Control Them at 1S1 , 1990 .

[25]  K. McCain.  The view from Garfield’s shoulders: Tri-citation mapping of Eugene Garfield’s citation image over three successive decades , 2010 .

[26]  Fiorenzo Franceschini,et al.  Research quality evaluation: comparing citation counts considering bibliometric database errors , 2015 .

[27]  Concepción S. Wilson,et al.  The Literature of Bibliometrics, Scientometrics, and Informetrics , 2001, Scientometrics.

[28]  魏屹东,et al.  Scientometrics , 2018, Encyclopedia of Big Data.

[29]  D. Nicholas,et al.  Scholarly reputation in the digital age and the role of emerging platforms and mechanisms , 2016 .

[30]  I. N. Sengupta.  Bibliometrics, Informetrics, Scientometrics and Librametrics: An Overview , 1992 .

[31]  Mike Thelwall,et al.  ResearchGate: Disseminating, communicating, and measuring Scholarship? , 2015, J. Assoc. Inf. Sci. Technol..

[32]  Benoît Godin,et al.  On the origins of bibliometrics , 2006, Scientometrics.

[33]  B. C. Brookes.  Biblio-, sciento-, infor-metrics?? what are we talking about ? , 1990 .

[34]  Blaise Cronin,et al.  The effect of postings information on searching behaviour Bibliometrics and beyond : some thoughts on web-based citation analysis , 2001 .

[35]  Henk F. Moed,et al.  Possible inaccuracies occurring in citation analysis , 1989, J. Inf. Sci..

[36]  Susanne Mikki,et al.  Digital Presence of Norwegian Scholars on Academic Network Sites—Where and Who Are They? , 2015, PloS one.

[37]  Mike Thelwall,et al.  Do highly cited researchers successfully use the social web? , 2014, Scientometrics.

[38]  S. M. Lawani.  Bibliometrics: Its Theoretical Foundations, Methods and Applications , 1981 .

[39]  Ronald Rousseau,et al.  Informetrics 87/88 : select proceedings of the First International Conference on Bibliometrics and Theoretical Aspects of Information Retrieval, Diepenbeek, Belgium, 25-28 August 1987 , 1988 .

[40]  Mike Thelwall,et al.  Which factors explain the Web impact of scientists' personal homepages? , 2007 .

[41]  Péter Jacsó,et al.  Dubious hit counts and cuckoo's eggs , 2006, Online Inf. Rev..

[42]  Mike Thelwall,et al.  Bibliometrics to webometrics , 2008, J. Inf. Sci..

[43]  Richard Van Noorden.  Online collaboration: Scientists and the social network , 2014, Nature.

[44]  Michael K. Buckland,et al.  Annual Review of Information Science and Technology , 2006, J. Documentation.

[45]  Nicolás Robinson-García,et al.  The Google scholar experiment: How to index false papers and manipulate bibliometric indicators , 2013, J. Assoc. Inf. Sci. Technol..

[46]  Miguel A. García-Pérez,et al.  Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: A case study for the computation of h indices in Psychology , 2010, J. Assoc. Inf. Sci. Technol..

[47]  Stacy Konkiel.  101 Innovations in Scholarly Communication – the Changing Research Workflow , 2015 .

[48]  Stephen J. Bensman.  Garfield and the impact factor , 2007 .

[49]  Judit Bar-Ilan,et al.  Coverage and adoption of altmetrics sources in the bibliometric community , 2014, Scientometrics.

[50]  Robert N. Broadus.  Toward a definition of “bibliometrics” , 1987, Scientometrics.

[51]  Fred R. Shapiro,et al.  Origins of Bibliometrics, Citation Indexing, and Citation Analysis: The Neglected Legal Literature , 1992, J. Am. Soc. Inf. Sci..

[52]  B. Peritz,et al.  On the Careers of Terminologies; the Case of Bibliometrics , 1984 .

[53]  Robert N. Broadus.  Early approaches to bibliometrics , 1987, J. Am. Soc. Inf. Sci..