Harnessing user library statistics for research evaluation and knowledge domain visualization

Social reference management systems provide a wealth of information that can be used for the analysis of science. In this paper, we examine whether user library statistics can produce meaningful results with regards to science evaluation and knowledge domain visualization. We are conducting two empirical studies, using a sample of library data from Mendeley, the world's largest social reference management system. Based on the occurrence of references in users' libraries, we perform a large-scale impact factor analysis and an exploratory co-readership analysis. Our preliminary findings indicate that the analysis of user library statistics can produce accurate, timely, and content-rich results. We find that there is a significant relationship between the impact factor and the occurrence of references in libraries. Using a knowledge domain visualization based on co-occurrence measures, we are able to identify two areas of topics within the emerging field of technology-enhanced learning.

[1]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[2]  Ian Rowlands,et al.  The missing link: journal usage metrics , 2007, Aslib Proc..

[3]  George Siemens.  Connectivism: A Learning Theory for the Digital Age , 2004 .

[4]  Alfred Kobsa,et al.  The Adaptive Web, Methods and Strategies of Web Personalization , 2007, The Adaptive Web.

[5]  Tom Murray,et al.  Authoring Intelligent Tutoring Systems: An analysis of the state of the art , 1999 .

[6]  James E. Driskell,et al.  Games, Motivation, and Learning: A Research and Practice Model , 2002 .

[7]  R. Sawyer.  The Cambridge Handbook of the Learning Sciences: Introduction , 2014 .

[8]  Peter Brusilovsky,et al.  Methods and techniques of adaptive hypermedia , 1996, User Modeling and User-Adapted Interaction.

[9]  Peter Brusilovsky,et al.  User Models for Adaptive Hypermedia and Adaptive Educational Systems , 2007, The Adaptive Web.

[10]  Bradley M. Hemminger,et al.  Scientometrics 2.0: New metrics of scholarly impact on the social Web , 2010, First Monday.

[11]  Les Carr,et al.  Trailblazing the literature of hypertext: author co-citation analysis (1989–1998) , 1999, HYPERTEXT '99.

[12]  Katherine W. McCain,et al.  Mapping authors in intellectual space: A technical overview , 1990, Journal of the American Society for Information Science.

[13]  Henry G. Small,et al.  Co-citation in the scientific literature: A new measure of the relationship between two documents , 1973, J. Am. Soc. Inf. Sci..

[14]  Katherine W. McCain,et al.  Mapping authors in intellectual space: A technical overview , 1990, J. Am. Soc. Inf. Sci..

[15]  Mike Thelwall,et al.  Bibliometrics to webometrics , 2008, J. Inf. Sci..

[16]  Alfred Kobsa.  User Modeling and User-Adapted Interaction , 2005, User Modeling and User-Adapted Interaction.

[17]  Johan Bollen,et al.  Mapping the structure of science through usage , 2006, Scientometrics.

[18]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[19]  Licia Calvi,et al.  AHA! An open Adaptive Hypermedia Architecture , 1998, New Rev. Hypermedia Multim..

[20]  David S. Goodsell,et al.  Automated docking using a Lamarckian genetic algorithm and an empirical binding free energy function , 1998, J. Comput. Chem..

[21]  Mu-Hsuan Huang,et al.  The relationships between the patent performance and corporation performance , 2012, J. Informetrics.

[22]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[23]  Peter Brusilovsky.  Methods and Techniques of Adaptive Hypermedia , 1996 .

[24]  M. Prensky.  Digital Natives, Digital Immigrants Part 1 , 2001 .

[25]  Sebastián Ventura,et al.  Educational data mining: A survey from 1995 to 2005 , 2007, Expert Syst. Appl..

[26]  M. Green,et al.  Digital natives , 2012, BDJ.

[27]  Anita Rosen.  e-Learning 2.0: Proven Practices and Emerging Technologies to Achieve Real Results , 2009 .

[28]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[29]  Natalia Stash,et al.  AHA! The adaptive hypermedia architecture , 2003, HYPERTEXT '03.

[30]  Ivana Marenzi,et al.  Who are you working with? Visualizing TEL Research Communities , 2010 .

[31]  Johan Bollen,et al.  MESUR: usage-based metrics of scholarly impact , 2007, JCDL '07.

[32]  M. Waldrop,et al.  Science 2.0. , 2008, Scientific American.

[33]  James A. Hendler,et al.  The Semantic Web" in Scientific American , 2001 .

[34]  Juan Gorraiz,et al.  Comparison of citation and usage indicators: the case of oncology journals , 2010, Scientometrics.

[35]  Peter Brusilovsky,et al.  Adaptive and Intelligent Web-based Educational Systems , 2003, Int. J. Artif. Intell. Educ..

[36]  Mike Thelwall,et al.  Validating online reference managers for scholarly impact measurement , 2011, Scientometrics.

[37]  Lokman I. Meho,et al.  Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar , 2007, J. Assoc. Inf. Sci. Technol..

[38]  Vannevar Bush,et al.  As we may think , 1945, INTR.

[39]  Victor Henning,et al.  Mendeley - A Last.fm For Research? , 2008, 2008 IEEE Fourth International Conference on eScience.

[40]  Peter Brusilovsky,et al.  Adaptive Hypermedia , 2001, User Modeling and User-Adapted Interaction.

[41]  Christoph Peylo,et al.  W2 - Adaptive and Intelligent Web-Based Education Systems , 2003, Intelligent Tutoring Systems.

[42]  Wilson C. Hsieh,et al.  Bigtable: A Distributed Storage System for Structured Data , 2006, TOCS.

[43]  A. Collins,et al.  Situated Cognition and the Culture of Learning , 1989 .

[44]  Marc Prensky,et al.  Digital game-based learning , 2000, CIE.

[45]  Huajun Chen,et al.  The Semantic Web , 2011, Lecture Notes in Computer Science.

[46]  Michael Zyda,et al.  From visual simulation to virtual reality to games , 2005, Computer.

[47]  Christoph Bartneck,et al.  Detecting h-index manipulation through self-citation analysis , 2010, Scientometrics.

[48]  Osmar R. Zaïane,et al.  Building a Recommender Agent for e-Learning Systems , 2002, ICCE.

[49]  B. E. Eckbo,et al.  Appendix , 1826, Epilepsy Research.

[50]  J. Benichou,et al.  Reading factor: a new bibliometric criterion for managing digital libraries. , 2002, Journal of the Medical Library Association : JMLA.

[51]  Tobias Siebenlist,et al.  Applying social bookmarking data to evaluate journal usage , 2011, J. Informetrics.

[52]  Mike Thelwall,et al.  Using the Web for research evaluation: The Integrated Online Impact indicator , 2010, J. Informetrics.

[53]  Daqing He,et al.  Social reference: aggregating online usage of scientific literature in CiteULike for clustering academic resources , 2011, JCDL '11.

[54]  Michael Granitzer,et al.  On generating large-scale ground truth datasets for the deduplication of bibliographic records , 2012, WIMS '12.

[55]  Iván Martínez-Ortiz,et al.  Educational game design for online education , 2008, Comput. Hum. Behav..

[56]  Stefanie N. Lindstaedt,et al.  Research Practices on the Web in the Field of Technology Enhanced Learning , 2011 .

[57]  Liang-Chu Chen,et al.  Using author co-citation analysis to examine the intellectual structure of e-learning: A MIS perspective , 2011, Scientometrics.

[58]  Stephen Downes.  E-learning 2.0 , 2005, ELERN.