Detection of dynamic execution errors in IBM system automation's rule-based expert system

We formally verify aspects of the rule-based expert system of IBM’s System Automation software for IBM’s zSeries mainframes. Starting with a formalization of the expert system in Propositional Dynamic Logic (PDL), we encode termination and determinism properties in PDL and its extension PDL. We then translate our decision problems to propositional logic and apply advanced SAT techniques for automated proofs. In order to locate real program bugs for each failed proof attempt, we apply extra formalization steps and represent propositional error formulae in concise normal form as Binary Decision Diagrams (BDDs). In our experiments, we revealed residual non-termination bugs in a tested program version close to shipment, and, after correcting them, we formally verified the absence of this class of bugs in the production code.

[1]  Bart Selman,et al.  Planning as Satisfiability , 1992, ECAI.

[2]  Derek L. Nazareth,et al.  Investigating the Applicability of Petri Nets for Rule-Based System Verification , 1993, IEEE Trans. Knowl. Data Eng..

[3]  Gruia-Catalin Roman,et al.  Applying formal verification methods to rule-based programs , 1994 .

[4]  David Harel,et al.  Looping vs. Repeating in Dynamic Logic , 1982, Inf. Control..

[5]  Alun D. Preece,et al.  Validating dynamic properties of rule-based systems , 1996, Int. J. Hum. Comput. Stud..

[6]  Anthony Hall,et al.  Seven myths of formal methods , 1990, IEEE Software.

[7]  Rose F. Gamble,et al.  A Theoretical Basis for the Assessment of Rule-Based System Reliability , 1999 .

[8]  J.F.A.K. van Benthem,et al.  Modal Correspondence Theory , 1977 .

[9]  Sharad Malik,et al.  Chaff: engineering an efficient SAT solver , 2001, Proceedings of the 38th Design Automation Conference (IEEE Cat. No.01CH37232).

[10]  D. Gabbay,et al.  Handbook of Philosophical Logic, Volume II. Extensions of Classical Logic , 1986 .

[11]  Mark E. Stickel,et al.  Proving Properties of Rule-Based Systems , 1992, Int. J. Softw. Eng. Knowl. Eng..

[12]  Frederick Hayes-Roth,et al.  Rule-based systems , 1985, CACM.

[13]  Derek L. Nazareth,et al.  Verification of rule-based knowledge using directed graphs , 1991 .

[14]  Mysore Ramaswamy,et al.  Using Directed Hypergraphs to Verity Rule-Based Expert Systems , 1997, IEEE Trans. Knowl. Data Eng..

[15]  Silvie Spreeuwenberg,et al.  A Knowledge Based Tool to Validate and Verify an Aion Knowledge Base , 1999, EUROVAV.

[16]  Jesse H. Poore.  Introduction to the special issue on: model-based statistical testing of software intensive systems , 2000, Inf. Softw. Technol..

[17]  Tomás E. Uribe,et al.  Ordered Binary Decision Diagrams and the Davis-Putnam Procedure , 1994, CCL.

[18]  Randal E. Bryant,et al.  Symbolic Boolean manipulation with ordered binary-decision diagrams , 1992, CSUR.

[19]  Bernhard K. Aichernig,et al.  Formal Specification of a Voice Communication System Used in Air Traffic Control , 1999, World Congress on Formal Methods.

[20]  Hantao Zhang,et al.  SATO: An Efficient Propositional Prover , 1997, CADE.

[21]  Armin Biere,et al.  Symbolic Model Checking without BDDs , 1999, TACAS.

[22]  Wolfgang Küchlin,et al.  PaSAT - Parallel SAT-Checking with Lemma Exchange: Implementation and Applications , 2001, Electron. Notes Discret. Math..

[23]  Dov M. Gabbay,et al.  Extensions of Classical Logic , 1989 .

[24]  Bart Selman,et al.  Encoding Plans in Propositional Logic , 1996, KR.

[25]  Wolfgang Küchlin,et al.  Towards a verification of the rule-based expert system of the IBM SA for OS/390 Automation Manager , 2001, Proceedings Second Asia-Pacific Conference on Quality Software.

[26]  Walton A. Perkins,et al.  Checking a Knowledge-Based System for Consistency and Completeness , 1987, AI Mag..

[27]  Mohan Tanniru,et al.  A Petri-Net Based Approach for Verifying the Integrity of Production Systems , 1992, Int. J. Man Mach. Stud..

[28]  Willem P. de Roever,et al.  A Proof Rule for Fair Termination of Guarded Commands , 1985, Inf. Control..

[29]  Andreas Kaiser.  A SAT-based propositional prover for consistency checking of automotive product data , 2001 .

[30]  Robert S. Streett,et al.  Propositional Dynamic Logic of Looping and Converse Is Elementarily Decidable , 1982, Inf. Control..

[31]  Wolfgang Küchlin,et al.  Proving Consistency Assertions for Automotive Product Data Management , 2000, Journal of Automated Reasoning.

[32]  Derek L. Nazareth,et al.  Issues in the Verification of Knowledge in Rule-Based Systems , 1989, Int. J. Man Mach. Stud..

[33]  Jonathan P. Bowen,et al.  Seven More Myths of Formal Methods , 1995, IEEE Softw..

[34]  Shari Lawrence Pfleeger,et al.  Investigating the Influence of Formal Methods , 1997, Computer.

[35]  Dan Craigen,et al.  Formal Methods Reality Check: Industrial Usage , 1993, IEEE Trans. Software Eng..

[36]  James A. Whittaker,et al.  A Markov Chain Model for Statistical Software Testing , 1994, IEEE Trans. Software Eng..