Key Issues Regarding Digital Libraries: Evaluation and Integration

This is the second book based on the 5S (Societies, Scenarios, Spaces, Structures, Streams) approach to digital libraries (DLs). Leveraging the first volume, on Theoretical Foundations, we focus on the key issues of evaluation and integration. These cross-cutting issues serve as a bridge for those interested in DLs, connecting the introduction and formal discussion in the first book, with the coverage of key technologies in the third book, and of illustrative applications in the fourth book. These two topics have central importance in the DL field, allowing it to be treated scientifically as well as practically. In the scholarly world, we only really understand something if we know how to measure and evaluate it. In the Internet era of distributed information systems, we only can be practical at scale if we integrate across both systems and their associated content. Evaluation of DLs must take place atmultiple levels,so we can address the different entities and their associated measures. Thus, for digital objects, we assess accessibility, pertinence, preservability, relevance, significance, similarity, and timeliness. Other measures are specific to higher-level constructs like metadata, collections, catalogs, repositories, and services. We tie these together through a case study of the 5SQual tool, which we designed and implemented to perform an automatic quantitative evaluation of DLs. Thus, across the Information Life Cycle, we describe metrics and software useful to assess the quality of DLs, and demonstrate utility with regard to representative application areas: archaeology and education. Though integration has been a challenge since the earliest work on DLs, we provide the first comprehensive 5S-based formal description of the DL integration problem, cast in the context of related work. Since archaeology is a fundamentally distributed enterprise, we describe ETANADL, for integrating Near Eastern Archeology sites and information. Thus, we show how 5S-based modeling can lead to integrated services and content. While the first book adopts a minimalist and formal approach to DLs, and provides a systematic and functional method to design and implement DL exploring services, here we broaden to practical DLs with richer metamodels, demonstrating the power of 5S for integration and evaluation. Table of Contents: Evaluation / Integration / Bibliography

[1]  Robert S. Taylor Question-Negotiation and Information Seeking in Libraries , 1968, Coll. Res. Libr..

[2]  Douglas J. Foskett A note on the concept of "relevance" , 1972, Inf. Storage Retr..

[3]  Phokion G. Kolaitis,et al.  Designing and refining schema mappings via data examples , 2011, SIGMOD '11.

[4]  Edward A. Fox,et al.  Leveraging OAI Harvesting To Disseminate Theses , 2003 .

[5]  Pedro M. Domingos,et al.  Learning to match ontologies on the Semantic Web , 2003, The VLDB Journal.

[6]  Edward A. Fox,et al.  Exploring the computing literature with visualization and stepping stones & pathways , 2006, Commun. ACM.

[7]  M. M. Kessler Bibliographic coupling between scientific papers , 1963 .

[8]  Kurt Maly,et al.  Arc - An OAI Service Provider for Digital Library Federation , 2001, D Lib Mag..

[9]  Richard Y. Wang,et al.  Data quality assessment , 2002, CACM.

[10]  Edward A. Fox,et al.  What Is a Successful Digital Library? , 2006, ECDL.

[11]  Richard Y. Wang,et al.  Anchoring data quality dimensions in ontological foundations , 1996, CACM.

[12]  Alberto H. F. Laender,et al.  BDBComp: building a digital library for the Brazilian computer science community , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..

[13]  Fernando Diaz,et al.  Integration of news content into web results , 2009, WSDM '09.

[14]  Sandra Payette,et al.  Interoperability for Digital Objects and Repositories: The Cornell/CNRI Experiments , 1999, D Lib Mag..

[15]  Ying Zhang Developing a holistic model for digital library evaluation , 2010 .

[16]  Henry G. Small,et al.  Co-citation in the scientific literature: A new measure of the relationship between two documents , 1973, J. Am. Soc. Inf. Sci..

[17]  E.A. Fox,et al.  ETANA-DL: managing complex information applications - an archaeology digital library , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..

[18]  Tefko Saracevic,et al.  Challenges for Digital Library Evaluation. , 2000 .

[19]  Ross Wilkinson,et al.  Preserving digital information forever , 2000, DL '00.

[20]  Gary Marchionini,et al.  Toward a worldwide digital library , 1998, CACM.

[21]  Berthier A. Ribeiro-Neto,et al.  A comparative study of citations and links in document classification , 2006, Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '06).

[22]  Raymond A. Lorie,et al.  A methodology and system for preserving digital data , 2002, JCDL '02.

[23]  Sudha Ram,et al.  Information systems interoperability: What lies beneath? , 2004, TOIS.

[24]  Hector Garcia-Molina,et al.  Archival storage for digital libraries , 1998, DL '98.

[25]  Wilhelm Hasselbring,et al.  Information system integration , 2000, CACM.

[26]  Edward A. Fox,et al.  Multilingual Federated Searching Across Heterogeneous Collections , 1998, D Lib Mag..

[27]  Edward A. Fox,et al.  An XML Log Standard and Tool for Digital Library Logging Analysis , 2002, ECDL.

[28]  Erhard Rahm,et al.  A survey of approaches to automatic schema matching , 2001, The VLDB Journal.

[29]  John A. N. Lee,et al.  PANEL on: Using CITIDEL as a Portal for IT Education , 2002 .

[30]  J. David Schloen Archaeological Data Models and Web Publication Using XML , 2001, Comput. Humanit..

[31]  Tefko Saracevic,et al.  RELEVANCE: A review of and a framework for the thinking on the notion in information science , 1997, J. Am. Soc. Inf. Sci..

[32]  Edward A. Fox,et al.  5SQual: a quality assessment tool for digital libraries , 2007, JCDL '07.

[33]  Nicholas J. Belkin,et al.  Braque: Design of an Interface to Support User Interaction in Information Retrieval , 1993, Inf. Process. Manag..

[34]  George W. Furnas,et al.  Considerations for information environments and the NaviQue workspace , 1998, DL '98.

[35]  Sudha Ram,et al.  Digital Libraries for the Next Millennium: Challenges and Research Directions , 1999, Inf. Syst. Frontiers.

[36]  Howard Greisdorf,et al.  Relevance thresholds: a multi-stage predictive model of how users evaluate information , 2003, Inf. Process. Manag..

[37]  Edward A. Fox,et al.  An OAI-Based Filtering Service for CITIDEL from NDLTD , 2003, ICADL.

[38]  Chris North,et al.  Citiviz: A Visual User Interface to the CITIDEL System , 2004, ECDL.

[39]  Martha Kyrillidou,et al.  Developing the DigiQUAL protocol for digital library evaluation , 2005, Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05).

[40]  Edward A. Fox,et al.  Prototyping Digital Libraries Handling Heterogeneous Data Sources - The ETANA-DL Case Study , 2004, ECDL.

[41]  Tefko Saracevic,et al.  Digital Library Evaluation: Toward Evolution of Concepts , 2000, Libr. Trends.

[42]  Henning Hopf Knowledge lost in information , 2007 .

[43]  Qinwei Zhu,et al.  5SGraph: A Modeling Tool for Digital Libraries , 2002 .

[44]  Edward A. Fox,et al.  Requirements Gathering and Modeling of Domain-Specific Digital Libraries with the 5S Framework: An Archaeological Case Study with ETANA , 2005, ECDL.

[45]  Michael Khoo,et al.  An Organizational Model for Digital Library Evaluation , 2011, TPDL.

[46]  Norbert Fuhr,et al.  Digital Libraries: A Generic Classification and Evaluation Scheme , 2001, ECDL.

[47]  D. A. Kemp Relevance, pertinence and information system development , 1974, Inf. Storage Retr..

[48]  Phokion G. Kolaitis,et al.  Learning schema mappings , 2012, ICDT '12.

[49]  Edward A. Fox,et al.  Visual Semantic Modeling of Digital Libraries , 2003, ECDL.

[50]  Renée J. Miller,et al.  Muse: Mapping Understanding and deSign by Example , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[51]  Gene Golovchinsky,et al.  Queries? Links? Is there a difference? , 1997, CHI.

[52]  Gary Marchionini,et al.  Evaluating Digital Libraries: A Longitudinal and Multifaceted View , 2000, Libr. Trends.

[53]  Wolf-Tilo Balke,et al.  Using Semantic Technologies in Digital Libraries - A Roadmap to Quality Evaluation , 2009, ECDL.

[54]  David M. Levy,et al.  Heroic measures: reflections on the possibility and purpose of digital preservation , 1998, DL '98.

[55]  Edward A. Fox,et al.  Streams, structures, spaces, scenarios, societies (5s): A formal model for digital libraries , 2004, TOIS.

[56]  S. Choudhury,et al.  A semi-automated digital preservation system based on semantic Web services , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..

[57]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[58]  Tapas Kanungo,et al.  Model characterization curves for federated search using click-logs: predicting user engagement metrics for the span of feasible operating points , 2011, WWW.

[59]  Kevin Chen-Chuan Chang,et al.  Interoperability for digital libraries worldwide , 1998, CACM.

[60]  Edward A. Fox,et al.  "What is a good digital library?" - A quality model for digital libraries , 2007, Inf. Process. Manag..

[61]  László Kovács,et al.  An experimental framework for comparative digital library evaluation: the logging scheme , 2006, Proceedings of the 6th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '06).

[62]  Edward A. Fox,et al.  Building quality into a digital library , 2000, DL '00.

[63]  Alexandru Balog,et al.  Testing a Multidimensional and Hierarchical Quality Assessment Model for Digital Libraries , 2011 .

[64]  Lillian N. Cassel,et al.  Using citidel as a portal for CS education , 2002 .

[65]  Peter Ingwersen,et al.  Dimensions of relevance , 2000, Inf. Process. Manag..

[66]  Giannis Tsakonas,et al.  An ontological representation of the digital library evaluation domain , 2011, J. Assoc. Inf. Sci. Technol..

[67]  Vijayalakshmi Atluri,et al.  SI in digital libraries , 2000, CACM.

[68]  Edward A. Fox,et al.  Integration of complex archeology digital libraries: An ETANA-DL experience , 2008, Inf. Syst..

[69]  Stefano Mizzaro,et al.  How many relevances in information retrieval? , 1998, Interact. Comput..

[70]  Georg Gottlob,et al.  Schema mapping discovery from data instances , 2010, JACM.

[71]  Getaneh Alemu,et al.  Integration of digital libraries and virtual learning environments: a literature review , 2009 .

[72]  Ellen M. Voorhees,et al.  Evaluation by highly relevant documents , 2001, SIGIR '01.

[73]  Sriram Raghavan,et al.  Search Middleware and the Simple Digital Library Interoperability Protocol , 2000, D Lib Mag..

[74]  Edward A. Fox,et al.  Development of a modern OPAC: from REVTOLC to MARIAN , 1993, SIGIR.

[75]  Giuseppina Vullo,et al.  Towards a Digital Library Policy and Quality Interoperability Framework: The DL.org Project , 2010 .

[76]  Edward A. Fox,et al.  The XML log standard for digital libraries: analysis, evolution, and deployment , 2003, 2003 Joint Conference on Digital Libraries, 2003. Proceedings..

[77]  Edward A. Fox,et al.  Schema mapper: a visualization tool for DL integration , 2005, Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05).

[78]  Carl Lagoze,et al.  Core services in the architecture of the national science digital library (NSDL) , 2002, JCDL '02.

[79]  Luis Gravano,et al.  Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection , 2002, VLDB.

[80]  Carol Peters,et al.  Evaluation of digital libraries , 2007, International Journal on Digital Libraries.

[81]  Herbert Van de Sompel,et al.  A Spectrum of Interoperability: The Site for Science Prototype for the NSDL , 2002, D Lib Mag..

[82]  James V. Hansen,et al.  Audit considerations in distributed processing systems , 1983, CACM.

[83]  Peter Gregor,et al.  Evaluating web resources for disability access , 2000, Assets '00.

[84]  Sebastian Ryszard Kruk,et al.  Semantic Digital Libraries , 2009, Semantic Digital Libraries.

[85]  Peter B. Danzig,et al.  The Harvest Information Discovery and Access System , 1995, Comput. Networks ISDN Syst..

[86]  Edward A. Fox,et al.  Incremental, Semi-automatic, Mapping-Based Integration of Heterogeneous Collections into Archaeological Digital Libraries: Megiddo Case Study , 2005, ECDL.

[87]  Rao Shen,et al.  Applying the 5S Framework To Integrating Digital Libraries , 2006 .

[88]  Edward A. Fox,et al.  Architecture of an expert system for composite document analysis, representation, and retrieval , 1997, Int. J. Approx. Reason..

[89]  Edward A. Fox,et al.  MARIAN: Flexible Interoperability for Federated Digital Libraries , 2001, ECDL.