Expert system validation through knowledge base refinement

Knowledge base (KB) refinement is a suitable technique to support expert system (ES) validation. When used for validation, KB refinement should be guided not only by the number of errors to solve but also by the importance of those errors. Most serious errors should be solved first, even causing other errors of lower importance but assuring a neat validity gain. These are the bases for IMPROVER, a KB refinement tool designed to support ES validation. IMPROVER refines ES for medical diagnosis with this classification of error importance: false negative > false positive > ordering mismatch. IMPROVER is being used to validate a real ES and some empirical results are given.

[1]  Edward H. Shortliffe,et al.  Rule Based Expert Systems: The Mycin Experiments of the Stanford Heuristic Programming Project (The Addison-Wesley series in artificial intelligence) , 1984 .

[2]  Carlos Sierra García Milord: arquitectura multi-nivell per a sistemes experts en classificacio , 1989 .

[3]  Philip Klahr,et al.  Evaluation of expert systems: issues and case studies , 1983 .

[4]  Pedro Meseguer,et al.  Verification of Multi-Level Rule-Based Expert Systems , 1991, AAAI.

[5]  Timothy Cain,et al.  The DUCTOR: A Theory Revision System for Propositional Domains , 1991, ML.

[6]  Derek H. Sleeman,et al.  Automating the Refinement of Knowledge-Based Systems , 1990, ECAI.

[7]  John Rushby,et al.  Quality Measures and Assurance for AI (Artificial Intelligence) Software , 1988 .

[8]  Paul J. Krause,et al.  Can we formally specify a medical decision support system? , 1993, IEEE Expert.

[9]  Sholom M. Weiss,et al.  Using Empirical Analysis to Refine Expert System Knowledge Bases , 1984, Artif. Intell..

[10]  Pedro Meseguer,et al.  Incremental Verification of Rule-Based Expert Systems , 1992, ECAI.

[11]  David C. Wilkins,et al.  On Debugging Rule Sets When Reasoning Under Uncertainty , 1986, AAAI.

[12]  Osman Balci,et al.  Validating Expert System Performance , 1987, IEEE Expert.

[13]  Raymond J. Mooney,et al.  Changing the Rules: A Comprehensive Approach to Theory Refinement , 1990, AAAI.

[14]  Allen Ginsberg,et al.  Theory Revision via Prior Operationalization , 1988, AAAI.

[15]  Marco Valtorta Some results on the computational complexity of refining confidence factors , 1991, Int. J. Approx. Reason..

[16]  Bruce G. Buchanan,et al.  The MYCIN Experiments of the Stanford Heuristic Programming Project , 1985 .

[17]  Sholom M. Weiss,et al.  SEEK2: A Generalized Approach to Automatic Knowledge Base Refinement , 1985, IJCAI.