Discovering the structure and impact of the digital library evaluation domain

The multidimensional nature of digital libraries evaluation domain poses several challenges to the research communities that intend to assess criteria, methods, products and tools, and also practice them. The amount of scientific production that is published in the domain hinders and disorientates the interested researchers. These researchers need guidance to exploit effectively the considerable amount of data and the diversity of methods, as well as to identify new research goals and develop their plans for future studies. This paper proposes a methodological pathway to investigate the core topics that structure the digital library evaluation domain and their impact. Further to the exploration of these topical entities, this study investigates also the researchers that contribute substantially to key topics, their communities and their relationships. The proposed methodology exploits topic modeling and network analysis in combination with citation and altmetrics analysis on a corpus consisting of the digital library evaluation papers presented in JCDL, ECDL/TDPL and ICADL conferences in the period 2001–2013.

[1]  Angela Elisabeth Smejkal,et al.  The knowledge domain of the academy of international business studies (AIB) conferences: a longitudinal scientometric perspective for the years 2006–2011 , 2013, Scientometrics.

[2]  George Adam Holland,et al.  Information science: an interdisciplinary effort? , 2008, J. Documentation.

[3]  Cassidy R. Sugimoto,et al.  LIS Dissertation Titles and Abstracts (1930–2009): Where Have All the Librar* Gone?1 , 2012, The Library Quarterly.

[4]  Peter Kraker,et al.  Altmetrics-based Visualizations Depicting the Evolution of a Knowledge Domain , 2014 .

[5]  Vincent Larivière,et al.  Conference proceedings as a source of scientific information: A bibliometric analysis , 2008, J. Assoc. Inf. Sci. Technol..

[6]  C. Haythornthwaite.  Social network analysis: An approach and technique for the study of information exchange☆ , 1996 .

[7]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[8]  Pan Jun Kim,et al.  Domain analysis with text mining: Analysis of digital library research trends using profiling methods , 2010, J. Inf. Sci..

[9]  Johan Bollen,et al.  Co-authorship networks in the digital library research community , 2005, Inf. Process. Manag..

[10]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[11]  Sarvnaz Karimi,et al.  Using Topic Models to Interpret MEDLINE's Medical Subject Headings , 2009, Australasian Conference on Artificial Intelligence.

[12]  Vincent Larivière,et al.  Tweets vs. Mendeley readers: How do these two social media metrics differ? , 2014, it Inf. Technol..

[13]  Nivio Ziviani,et al.  Assessing the quality of scientific conferences based on bibliographic citations , 2010, Scientometrics.

[14]  Michalis Sfakakis,et al.  Charting the digital library evaluation domain with a semantically enhanced mining methodology , 2013, JCDL '13.

[15]  Christine L. Borgman,et al.  What are Digital Libraries? Competing Visions , 1999, Inf. Process. Manag..

[16]  Tefko Saracevic,et al.  Digital Library Evaluation: Toward Evolution of Concepts , 2000, Libr. Trends.

[17]  Richard Furuta,et al.  Do altmetrics follow the crowd or does the crowd follow altmetrics? , 2014, IEEE/ACM Joint Conference on Digital Libraries.

[18]  C. Lee Giles,et al.  A generalized topic modeling approach for automatic document annotation , 2015, International Journal on Digital Libraries.

[19]  Elisa Bellotti,et al.  Getting funded. Multi-level network of physicists in Italy , 2012, Soc. Networks.

[20]  Padhraic Smyth,et al.  Subject metadata enrichment using statistical topic models , 2007, JCDL '07.

[21]  I. Kron.  Getting funded. , 2000, The Journal of thoracic and cardiovascular surgery.

[22]  Cassidy R. Sugimoto,et al.  The cognitive structure of Library and Information Science: Analysis of article title words , 2011, J. Assoc. Inf. Sci. Technol..

[23]  C. Lee Giles,et al.  Finding topic trends in digital libraries , 2009, JCDL '09.

[24]  Michalis Sfakakis,et al.  Discovering the Topical Evolution of the Digital Library Evaluation Community , 2015, MTSR.

[25]  Edward A. Fox,et al.  Curriculum development for digital libraries , 2006, Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '06).

[26]  Krishna P. Gummadi,et al.  Measurement and analysis of online social networks , 2007, IMC '07.

[27]  L. Freeman.  Centrality in social networks conceptual clarification , 1978 .

[28]  Christopher J. Fox,et al.  A stop list for general text , 1989, SIGF.

[29]  Virender Singh,et al.  Analyzing worldwide research in hardware architecture, 1997--2011 , 2014, Commun. ACM.

[30]  Rafael Aleixandre-Benavent,et al.  Coauthorship and Institutional Collaborations on Cost-Effectiveness Analyses: A Systematic Network Analysis , 2012, PloS one.

[31]  John Skvoretz,et al.  Node centrality in weighted networks: Generalizing degree and shortest paths , 2010, Soc. Networks.

[32]  Roland H. C. Yap,et al.  Experiments on Rating Conferences with CORE and DBLP , 2014, D Lib Mag..

[33]  Timothy Baldwin,et al.  Evaluating topic models for digital libraries , 2010, JCDL '10.

[34]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[35]  Juan-Zi Li,et al.  Conference Mining via Generalized Topic Modeling , 2009, ECML/PKDD.

[36]  Min Song,et al.  Analyzing topic evolution in bioinformatics: investigation of dynamics of the field with conference data in DBLP , 2014, Scientometrics.

[37]  Giannis Tsakonas,et al.  An exploration of the research trends in the digital library evaluation domain , 2012, JCDL '12.

[38]  Gobinda G. Chowdhury,et al.  Digital Library Research (1990-2010): A Knowledge Map of Core Topics and Subtopics , 2011, ICADL.

[39]  Mark Stevenson,et al.  Computing similarity between items in a digital library of cultural heritage , 2013, JOCCH.

[40]  David Newman,et al.  Are learned topics more useful than subject headings , 2011, JCDL '11.

[41]  Giannis Tsakonas,et al.  Where and how knowledge on digital library evaluation spreads: a case study on conference literature , 2014 .