Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows

Although there is evidence that counting the readers of an article in the social reference site, Mendeley, may help to capture its research impact, the extent to which this is true for different scientific fields is unknown. In this study, we compare Mendeley readership counts with citations for different social sciences and humanities disciplines. The overall correlation between Mendeley readership counts and citations for the social sciences was higher than for the humanities. Low and medium correlations between Mendeley bookmarks and citation counts in all the investigated disciplines suggest that these measures reflect different aspects of research impact. Mendeley data were also used to discover patterns of information flow between scientific fields. Comparing information flows based on Mendeley bookmarking data and cross‐disciplinary citation analysis for the disciplines revealed substantial similarities and some differences. Thus, the evidence from this study suggests that Mendeley readership data could be used to help capture knowledge transfer across scientific disciplines, especially for people that read but do not author articles, as well as giving impact evidence at an earlier stage than is possible with citation counts.

[1]  Sydney J. Pierce,et al.  Boundary Crossing in Research Literatures as a Means of Interdisciplinary Information Transfer , 1999, J. Am. Soc. Inf. Sci..

[2]  Judit Bar-Ilan.  JASIST 2001–2010 , 2012 .

[3]  Paul Ginsparg,et al.  Positional effects on citation and readership in arXiv , 2009, J. Assoc. Inf. Sci. Technol..

[4]  Thed N. van Leeuwen,et al.  Citation delay in interdisciplinary knowledge exchange , 2001, Scientometrics.

[5]  Paul Groth,et al.  The Altmetrics Collection , 2012, PloS one.

[6]  Stephen S. Murray,et al.  The bibliometric properties of article readership information , 2005, J. Assoc. Inf. Sci. Technol..

[7]  M. Thelwall,et al.  F 1000 , Mendeley and Traditional Bibliometric Indicators , 2012 .

[8]  Svein Kyvik,et al.  Changing trends in publishing behaviour among university faculty, 1980-2000 , 2003, Scientometrics.

[9]  G. Stevens.  An Alliance Confirmed Planning Literature and the Social Sciences , 1990 .

[10]  Anton J. Nederhof,et al.  Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review , 2006, Scientometrics.

[11]  Ian Rowlands,et al.  The missing link: journal usage metrics , 2007, Aslib Proc..

[12]  Johan Bollen,et al.  A Principal Component Analysis of 39 Scientific Impact Measures , 2009, PloS one.

[13]  Wolfgang G. Stock,et al.  Impact and relevance of LIS journals: A scientometric analysis of international and German-language LIS journals - Citation analysis versus reader survey , 2004, J. Assoc. Inf. Sci. Technol..

[14]  Donald A. Krueckeberg.  The tuition of American planning: from dependency toward self-reliance , 1985 .

[15]  Tobias Siebenlist,et al.  Applying social bookmarking data to evaluate journal usage , 2011, J. Informetrics.

[16]  Blaise Cronin,et al.  The export of ideas from information science , 1990, J. Inf. Sci..

[17]  Peter Kraker,et al.  Harnessing user library statistics for research evaluation and knowledge domain visualization , 2012, WWW.

[18]  Mike Thelwall,et al.  Validating online reference managers for scholarly impact measurement , 2011, Scientometrics.

[19]  Fernando González-Ladrón-de-Guevara,et al.  Uses of explicit and implicit tags in social bookmarking , 2012, J. Assoc. Inf. Sci. Technol..

[20]  L. Butler,et al.  Explaining Australia’s increased share of ISI publications—the effects of a funding formula based on publication counts , 2003 .

[21]  Johan Bollen,et al.  Toward alternative metrics of journal impact: A comparison of download and citation data , 2005, Inf. Process. Manag..

[22]  Johan Bollen,et al.  Usage bibliometrics , 2011, Annu. Rev. Inf. Sci. Technol..

[23]  Gunther Eysenbach,et al.  Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact , 2011, Journal of medical Internet research.

[24]  Yu-Wei Chang,et al.  Characteristics of research output in social sciences and humanities: From a research evaluation perspective , 2008, J. Assoc. Inf. Sci. Technol..

[25]  Lokman I. Meho,et al.  The shifting balance of intellectual trade in information studies , 2008, J. Assoc. Inf. Sci. Technol..

[26]  Daqing He,et al.  Social reference: aggregating online usage of scientific literature in CiteULike for clustering academic resources , 2011, JCDL '11.

[27]  Abagail McWilliams,et al.  The Balance of Trade Between Disciplines , 2005 .

[28]  Schubert Foo,et al.  Perspectives on social tagging , 2009, J. Assoc. Inf. Sci. Technol..

[29]  D. Blecic.  Measurements of journal use: an analysis of the correlations between three methods. , 1999, Bulletin of the Medical Library Association.

[30]  Juan Gorraiz,et al.  Comparison of citation and usage indicators: the case of oncology journals , 2010, Scientometrics.

[31]  Mike Thelwall.  Journal impact evaluation: a webometric perspective , 2012, Scientometrics.

[32]  Bradley M. Hemminger,et al.  Scientometrics 2.0: New metrics of scholarly impact on the social Web , 2010, First Monday.

[33]  Rong Tang,et al.  Evolution of the interdisciplinary characteristics of information and library science , 2005, ASIST.

[34]  Bradley M. Hemminger,et al.  Altmetrics in the wild: Using social media to explore scholarly impact , 2012, ArXiv.

[35]  Hiroaki Urata,et al.  Information flows among academic disciplines in Japan , 1990, Scientometrics.

[36]  Rolf A. Zwaan,et al.  Assessing the usefulness of bibliometric indicators for the humanities and the social and beha vioural sciences: A comparative study , 1989, Scientometrics.

[37]  Chris Armbruster.  Access, Usage and Citation Metrics: What Function for Digital Libraries and Repositories in Research Evaluation? , 2008 .

[38]  Ian Rowlands,et al.  Scholarly communication in the digital environment: The 2005 survey of journal author behaviour and attitudes , 2005, Aslib Proc..

[39]  Stevan Harnad,et al.  Earlier Web Usage Statistics as Predictors of Later Citation Impact , 2005, J. Assoc. Inf. Sci. Technol..

[40]  Wolfgang Glänzel,et al.  Tracing the role of individual journals in a cross-citation network based on different indicators , 2009, Scientometrics.

[41]  Vincent Larivière.  The historical evolution of interdisciplinarity: 1900-2008 , 2010 .

[42]  Eduard Jan Rinia,et al.  Measurement and evaluation of interdisciplinary research and knowledge transfer , 2007 .

[43]  Thed N. van Leeuwen,et al.  Measuring knowledge transfer between fields of science , 2002, Scientometrics.

[44]  María Bordons,et al.  Analysis of Cross-Disciplinary Research Through Bibliometric Tools , 2004 .

[45]  Robert L. Goldstone,et al.  The Import and Export of Cognitive Science , 2006, Cogn. Sci..

[46]  Cornelius Le Pair,et al.  Switching between academic disciplines in universities in the Netherlands , 1980, Scientometrics.

[47]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[48]  Karla Hahn,et al.  Evaluative Usage-Based Metrics for the Selection of E-Journals , 2002 .

[49]  Judit Bar-Ilan,et al.  Beyond citations: Scholars' visibility on the social Web , 2012, ArXiv.

[50]  Isabel Gómez,et al.  Interdisciplinarity in science: A tentative typology of disciplines and research areas , 2003, J. Assoc. Inf. Sci. Technol..

[51]  Mike Thelwall,et al.  Variations between subjects in the extent to which the social sciences have become more interdisciplinary , 2011, J. Assoc. Inf. Sci. Technol..

[52]  C. Neylon,et al.  Article-Level Metrics and the Evolution of Scientific Impact , 2009, PLoS biology.

[53]  Henk F. Moed,et al.  Citation Analysis in Research Evaluation , 1899 .

[54]  Crystal Fulton.  A decade of SIG/USE: Celebrating SIG/USE and information behavior research: Introduction , 2010 .

[55]  James D. Neeley,et al.  The management and social science literatures: An interdisciplinary cross-citation analysis , 1981, J. Am. Soc. Inf. Sci..

[56]  Ismael Rafols,et al.  Global maps of science based on the new Web-of-Science categories , 2012, Scientometrics.

[57]  Jenny Fry,et al.  Measuring researchers’ use of scholarly information through social bookmarking data: A case study of BibSonomy , 2012, J. Inf. Sci..

[58]  Juan Gorraiz,et al.  Global usage versus global citation metrics: The case of pharmacology journals , 2011, J. Assoc. Inf. Sci. Technol..