"What is a good digital library?" - A quality model for digital libraries

In this article, we elaborate on the meaning of quality in digital libraries (DLs) by proposing a model that is deeply grounded in a formal framework for digital libraries: 5S (Streams, Structures, Spaces, Scenarios, and Societies). For each major DL concept in the framework we formally define a number of dimensions of quality and propose a set of numerical indicators for those quality dimensions. In particular, we consider key concepts of a minimal DL: catalog, collection, digital object, metadata specification, repository, and services. Regarding quality dimensions, we consider: accessibility, accuracy, completeness, composability, conformance, consistency, effectiveness, efficiency, extensibility, pertinence, preservability, relevance, reliability, reusability, significance, similarity, and timeliness. Regarding measurement, we consider characteristics like: response time (with regard to efficiency), cost of migration (with respect to preservability), and number of service failures (to assess reliability). For some key DL concepts, the (quality dimension, numerical indicator) pairs are illustrated through their application to a number of ''real-world'' digital libraries. We also discuss connections between the proposed dimensions of DL quality and an expanded version of a workshop's consensus view of the life cycle of information in digital libraries. Such connections can be used to determine when and where quality issues can be measured, assessed, and improved - as well as how possible quality problems can be prevented, detected, and eliminated.

[1]  Ellen M. Voorhees,et al.  Evaluation by highly relevant documents , 2001, SIGIR '01.

[2]  Giorgio Maria Di Nunzio,et al.  Evaluation of a digital library system , 2004 .

[3]  Douglas J. Foskett.  A note on the concept of "relevance" , 1972, Inf. Storage Retr..

[4]  Richard Y. Wang,et al.  Data quality assessment , 2002, CACM.

[5]  Christine L. Borgman,et al.  Social aspects of digital libraries (working session) , 1996, DL '96.

[6]  Edward A. Fox,et al.  Open digital libraries , 2002 .

[7]  Ross Wilkinson,et al.  Preserving digital information forever , 2000, DL '00.

[8]  Hector Garcia-Molina,et al.  Archival storage for digital libraries , 1998, DL '98.

[9]  Stefano Mizzaro,et al.  How many relevances in information retrieval? , 1998, Interact. Comput..

[10]  Khalid Sayood,et al.  Introduction to Data Compression , 1996 .

[11]  Edward A. Fox,et al.  An OAI-Based Filtering Service for CITIDEL from NDLTD , 2003, ICADL.

[12]  Luis Gravano,et al.  Distributed Search over the Hidden Web: Hierarchical Database Sampling and Selection , 2002, VLDB.

[13]  Alexander T. Nicolai,et al.  A Note on the Concept of Relevance , 2007 .

[14]  Richard Y. Wang,et al.  Anchoring data quality dimensions in ontological foundations , 1996, CACM.

[15]  Earl R. Babbie,et al.  The practice of social research , 1969 .

[16]  Jane Hunter,et al.  Implementing Preservation Strategies for Complex Multimedia Objects , 2003, ECDL.

[17]  Thomas C. Redman,et al.  Data Quality Management and Technology , 1992 .

[18]  Howard Greisdorf,et al.  Relevance thresholds: a multi-stage predictive model of how users evaluate information , 2003, Inf. Process. Manag..

[19]  S. Mizzaro,et al.  A Cognitive Analysis of Information Retrieval , 1996 .

[20]  M. M. Kessler.  Bibliographic coupling between scientific papers , 1963 .

[21]  Christine L. Borgman,et al.  Workshop report: Social Aspects Of Digital Libraries. , 1997 .

[22]  David M. Levy,et al.  Heroic measures: reflections on the possibility and purpose of digital preservation , 1998, DL '98.

[23]  Edward A. Fox,et al.  Streams, structures, spaces, scenarios, societies (5s): A formal model for digital libraries , 2004, TOIS.

[24]  Norbert Fuhr,et al.  Digital Libraries: A Generic Classification and Evaluation Scheme , 2001, ECDL.

[25]  Mimi Recker,et al.  Understanding educator perceptions of "quality" in digital libraries , 2003, 2003 Joint Conference on Digital Libraries, 2003. Proceedings..

[26]  D. A. Kemp.  Relevance, pertinence and information system development , 1974, Inf. Storage Retr..

[27]  Henry G. Small,et al.  Co-citation in the scientific literature: A new measure of the relationship between two documents , 1973, J. Am. Soc. Inf. Sci..

[28]  Tefko Saracevic,et al.  RELEVANCE: A review of and a framework for the thinking on the notion in information science , 1997, J. Am. Soc. Inf. Sci..

[29]  Jeff Rothenberg.  Using Emulation to Preserve Digital Documents , 2000 .

[30]  Jane Hunter,et al.  A semi-automated digital preservation system based on semantic web services , 2004, JCDL.

[31]  James V. Hansen,et al.  Audit considerations in distributed processing systems , 1983, CACM.

[32]  Peter Gregor,et al.  Evaluating web resources for disability access , 2000, Assets '00.

[33]  Raymond A. Lorie,et al.  Long term preservation of digital information , 2001, JCDL '01.

[34]  Edward A. Fox,et al.  The Core: Digital Library Education in Library and Information Science Programs , 2006, D Lib Mag..

[35]  Peter Ingwersen,et al.  Dimensions of relevance , 2000, Inf. Process. Manag..

[36]  Birger Larsen,et al.  ACM SIGIR 2004 Workshop on "Information Retrieval in Context" , 2004 .

[37]  Donna Harman,et al.  Information Processing and Management , 2022 .

[38]  Edward A. Fox,et al.  Prototyping Digital Libraries Handling Heterogeneous Data Sources - The ETANA-DL Case Study , 2004, ECDL.

[39]  Tefko Saracevic,et al.  Digital Library Evaluation: Toward Evolution of Concepts , 2000, Libr. Trends.

[40]  S. Choudhury,et al.  A semi-automated digital preservation system based on semantic Web services , 2004, Proceedings of the 2004 Joint ACM/IEEE Conference on Digital Libraries, 2004..