Towards effective evaluation of digital community information systems

Digital Community Information Systems (CIS) are increasing in both size and scale. These systems help people obtain community information (CI) for the myriad situations that arise in everyday life. As these systems migrate to the online environment, increasing complexities and scale provide the impetus for improvements in system design. System re-design, however, relies on the effective evaluation of digital CIS. Despite the plethora of evaluation frameworks from the systems, organizational, and information retrieval fields, minimal research has directly addressed the issue of digital CIS evaluation. Following a selective review of digital CIS and evaluation approaches, a layered evaluation framework is proposed to guide the evaluation of digital CIS.

[1]  William S. Cooper,et al.  On selecting a measure of retrieval effectiveness , 1973, J. Am. Soc. Inf. Sci..

[2]  Scott Hamilton,et al.  Evaluating Information Systems Effectiveness - Part I: Comparing Evaluation Approaches , 1981, MIS Q..

[3]  Prudence W. Dalrymple,et al.  Users' Experience of Information Retrieval Systems: An Exploration of the Relationship between Search Experience and Affective Measures. , 1992 .

[4]  Gary Marchionini,et al.  Evaluating hypermedia and learning: methods and results from the Perseus Project , 1994, TOIS.

[5]  B M Wildemuth,et al.  End-user search behaviors and their relationship to search effectiveness. , 1995, Bulletin of the Medical Library Association.

[6]  Wolfgang Dzida.  International usability standards , 1996, CSUR.

[7]  K. Pettigrew.  Nurses' perceptions of their needs for community information : Results of an exploratory study in Southwestern Ontario , 1996 .

[8]  Steve Smithson,et al.  The management of change for information systems evaluation practice: Experience from a case study , 1996 .

[9]  Matthew L. Saxton,et al.  Reference Service Evaluation and Meta-Analysis: Findings and Methodological Issues , 1997, The Library Quarterly.

[10]  Ann Peterson Bishop,et al.  Logins and Bailouts: Measuring Access, Use, and Success in Digital Libraries , 1998 .

[11]  Christine L. Borgman,et al.  What are Digital Libraries? Competing Visions , 1999, Inf. Process. Manag..

[12]  Tefko Saracevic,et al.  Digital Library Evaluation: Toward Evolution of Concepts , 2000, Libr. Trends.

[13]  Gary Marchionini,et al.  Evaluating Digital Libraries: A Longitudinal and Multifaceted View , 2000, Libr. Trends.

[14]  Ann Peterson Bishop,et al.  Digital libraries: situating use in changing information infrastructure , 2000 .