Validating dynamic properties of rule-based systems

Abstract Rule-based systems can be viewed as possessing two sets of properties: static and dynamic. Static properties are those that can be evaluated without executing the system, and dynamic properties can be evaluated only by examining how the system operates at run time. The dynamic properties of a rule-based system have been largely neglected in validation and verification work done thus far. Structural verification and static testing techniques do not yield information on how a rule-based system achieves its goals at run-time, given a set of input data. This paper presents a model for the relationship between the goal states achieved by a rule-based system, the set of inter-related rules that must fire to achieve each goal state, and the data items required for the rules in the rule sequence to fire. Then, we describe a method for applying this model to study the dynamic properties of a rule-based system. It is demonstrated that this model permits the validation of dynamic properties of a rule-based system, enabling system developers to decide: (1) if the manner in which the system pursues goals is valid according to the specifications (and expectations) of the designers; (2) what relationship exists between the quality of system output for a given test case and the goals achieved during problem-solving on that test case; and (3) how the overall problem-solving activity of the system relates to the availability of input data.

[1]  Peter Grogono,et al.  Specifying an expert system , 1991 .

[2]  Frederick Hayes-Roth,et al.  Rule-based systems , 1985, CACM.

[3]  Wei-Tek Tsai,et al.  Comparative evaluation of expert system testing methods , 1992, Proceedings Fourth International Conference on Tools with Artificial Intelligence TAI '92.

[4]  Thomas J. Laffey,et al.  Real-Time Knowledge-Based Systems , 1988, AI Mag..

[5]  Alun Preece,et al.  Verifying expert systems: A logical framework and a practical tool , 1992 .

[6]  Osman Balci,et al.  Validating Expert System Performance , 1987, IEEE Expert.

[7]  Luc Steels,et al.  Components of Expertise , 1990, AI Mag..

[8]  William J. Clancey,et al.  Model Construction Operators , 1992, Artif. Intell..

[9]  Alun D. Preece.  Verification, Validation, and Test of Knowledge-Based Systems , 1992, AI Mag..

[10]  Pedro Meseguer,et al.  Verification of Multi-Level Rule-Based Expert Systems , 1991, AAAI.

[11]  Alun D. Preece,et al.  Exploring the Structure of Rule Based Systems , 1993, AAAI.

[12]  Keith E. Williamson,et al.  Inconsistency and redundancy checking for quasi-first-order-logic knowledge bases , 1993 .

[13]  Thiruvengadam Radhakrishnan,et al.  Towards a tool for the design of cooperating expert systems , 1992, Proceedings Fourth International Conference on Tools with Artificial Intelligence TAI '92.

[14]  Uma G. Gupta.  Validating and Verifying Knowledge-Based Systems , 1991 .

[15]  John C. Cherniavsky,et al.  Validation, Verification, and Testing of Computer Software , 1982, CSUR.

[16]  Bob J. Wielinga,et al.  KADS: a modelling approach to knowledge engineering , 1992 .

[17]  James D. Kiper.  Structural testing of rule-based expert systems , 1992, TSEM.

[18]  T. Radhakrishnan,et al.  Validation of an expert system intended for research in distributed artificial intelligence , 1991 .

[19]  Leonard Adelman,et al.  Experiments, quasi-experiments, and case studies: A review of empirical methods for evaluating decision support systems , 1991, IEEE Trans. Syst. Man Cybern..

[20]  Clifford Grossner.  Models and tools for cooperating rule-based systems , 1996 .

[21]  Lance A. Miller,et al.  Dynamic testing of knowledge bases using the heuristic testing approach , 1990 .

[22]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[23]  Derek L. Nazareth,et al.  Investigating the Applicability of Petri Nets for Rule-Based System Verification , 1993, IEEE Trans. Knowl. Data Eng..

[24]  Daniel O’Leary.  Verification, Validation, and Test of Knowledge-Based Systems , 1996 .

[25]  Alun Preece,et al.  An effective logical framework for knowledge-based systems verification , 1994 .

[26]  Herbert A. Simon,et al.  The Structure of Ill Structured Problems , 1973, Artif. Intell..