Challenges for Digital Library Evaluation.

While there were many efforts in research and practice of digital libraries, evaluation was not a conspicuous activity. It is well recognized that digital library evaluation is a complex and difficult undertaking. We enumerate the challenges facing digital library evaluation and suggest a conceptual framework for evaluation, A review of evaluation efforts in research and practice concentrates on derivation of criteria used in evaluation. Essential requirements for evaluation are stated. Discussed are constructs, context and criteria of digital libraries: What should we evaluate? For what purpose do we evaluate? Who should evaluate? At what level do we evaluate? Upon what criteria do we evaluate? In addition we include suggestions for adaptation of criteria from related activities. The paper is considered as a part of the evolution of concepts for digital library evaluation.

[1]  J. Farradane,et al.  Information Science , 1971, Nature.

[2]  Terence R. Smith,et al.  Alexandria digital library: user evaluation studies and system design , 2000, J. Am. Soc. Inf. Sci..

[3]  Christine L. Borgman,et al.  What are Digital Libraries? Competing Visions , 1999, Inf. Process. Manag..

[4]  Gary Marchionini,et al.  Evaluating hypermedia and learning: methods and results from the Perseus Project , 1994, TOIS.

[5]  Gary Marchionini,et al.  The People in Digital Libraries: Multifaceted Approaches to Assessing Needs and Impact , 1999 .

[6]  Terence R. Smith,et al.  Alexandria digital library: user evaluation studies and system design , 2000 .

[7]  T. G.,et al.  Logic in Practice , 1934, Nature.

[8]  Maria Bonn,et al.  A Report on the PEAK Experiment: Context and Design , 1999, D-Lib Magazine.

[9]  Michael E. Lesk,et al.  Practical Digital Libraries: Books, Bytes, and Bucks , 1997 .

[10]  Maria Bonn,et al.  A Report on the PEAK Experiment: Usage and Economic Behavior , 1999, D Lib Mag..

[11]  Paul B. Kantor,et al.  Studying the Value of Library and Information Services. Part I. Establishing a Theoretical Framework , 1997, J. Am. Soc. Inf. Sci..

[12]  Paul B. Kantor,et al.  Studying the Value of Library and Information Services. Part I: Establishing a Theoretical Framework. , 1997 .

[13]  Barbara P. Buttenfield,et al.  Usability Evaluation of Digital Libraries , 1999 .

[14]  Ann Peterson Bishop,et al.  Document Structure and Digital Libraries: How Researchers Mobilize Information in Journal Articles , 1999, Inf. Process. Manag..

[15]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[16]  Louise T. Su.  Evaluation Measures for Interactive Information Retrieval , 1992, Inf. Process. Manag..

[17]  Kathleen M. Heim.  If you want to evaluate your library , 1990, J. Am. Soc. Inf. Sci..

[18]  Nancy A. Van House,et al.  Understanding complex information environments: a social analysis of watershed planning , 1997, DL '97.

[19]  Jamie McKenzie,et al.  Libraries of the Future , 1996 .