Facilitating the Practical Evaluation of Organizational Memories Using the Goal-Question-Metric Technique

It is an important industrial need to deliver high-quality knowledge-based systems and organizational memories (e.g., to support service management or knowledge management in general). Evaluation is required to ensure this high quality and guide the development and maintenance. We present an approach for facilitating practical evaluation of organizational memories that meets the requirements for good measurements in knowledge engineering. The base of this methodology is the GoalQuestion-Metric (GQM) technique, which is an industrial-strength technique for goaloriented measurement and evaluation from the field of software engineering. The practical benefit of GQM is demonstrated by a case study where GQM was applied to an existing case-based reasoning system/application.

[1]  Victor R. Basili,et al.  The Experimental Paradigm in Software Engineering , 1992, Experimental Software Engineering Issues.

[2]  Klaus-Dieter Althoff,et al.  Evaluating Case-Based Reasoning Systems , 1995 .

[3]  H. D. Rombach,et al.  THE EXPERIENCE FACTORY , 1999 .

[4]  Victor R. Basili,et al.  The Experience Factory and its Relationship to Other Improvement Paradigms , 1993, ESEC.

[5]  C. Prahalad,et al.  The Core Competence of the Corporation , 1990 .

[6]  S.J.J. Smith,et al.  Empirical Methods for Artificial Intelligence , 1995 .

[7]  Lorne Olfman,et al.  Organizational Memory , 1998, Proceedings of the Thirty-First Hawaii International Conference on System Sciences.

[8]  Klaus-Dieter Althoff,et al.  Learning From and Reasoning About Case-based Reasoning Systems , 1997, XPS.

[9]  Klaus-Dieter Althoff,et al.  Decision Support for Case-Based Applications , 1996, Wirtsch..

[10]  Tim Menzies.  Evaluation Issues for Problem Solving Methods , 1998 .

[11]  Carsten Tautz,et al.  Practical Evaluation of an Organizational Memory Using the Goal-Question-Metric Technique , 1999, XPS.

[12]  Enrico Motta,et al.  Making knowledge engineering technology work , 1997 .

[13]  Susanne Kirchhoff-Kestel.  Abbildungsqualität von wissensbasierten Systemen: eine Methodologie zur Evaluierung , 1994 .

[14]  William S. Cooper,et al.  On selecting a measure of retrieval effectiveness , 1973, J. Am. Soc. Inf. Sci..

[15]  Agnar Aamodt,et al.  Relating Case-Based Problem Solving and Learning Methods to Task and Domain Characteristics: Towards an Analytic Framework , 1996, AI Commun..

[16]  K. Althoff.  Case-Based Reasoning for Decision Support and Diagnostic Problem Solving: The INRECA Approach , 1995 .

[17]  Lionel C. Briand,et al.  Practical guidelines for measurement-based process improvement , 1996, Softw. Process. Improv. Pract..

[18]  Norman E. Fenton,et al.  Software Metrics: A Rigorous Approach , 1991 .

[19]  Tim Menzies.  Evaluation Issues with Critical Success Metrics , 1998 .

[20]  Carsten Tautz,et al.  Improving Organizational Memories Through User Feedback , 2000 .

[21]  Sean Breen,et al.  Developing Industrial Case-Based Reasoning Applications: The INRECA Methodology , 1999 .