System Refinement in Practice - Using a Formal Method to Modify Real-Life Knowledge

The pros and cons of formal methods are the subject of many discussions in Artificial Intelligence (AI). Here, the authors describe a formal method that aims at system refinement based on the results of a test case validation technology for rule-based systems. This technique provides sufficient information to estimate the validity of each single rule. Validity in this context is estimated by evaluating the test cases that used the considered rule. The objective is to overcome the particular invalidities that are revealed by the validation process. System refinement has to be set into the context of “learning by examples”. Classical approaches are often not useful for system refinement in practice. They often lead to a knowledge base containing rules that are difficult to interpret by domain experts. The refinement process presented here is characterized by (1) using human expertise that also is a product of the validation technique and (2) keeping as much as possible of the (original) knowledge base. This is a way to avoid the drawbacks of other approaches and to enjoy the benefits of formal methods nevertheless. The validation process provides “better solutions” for test cases that have a solution which received a bad validity assessment by the validating experts. This knowledge is utilized by a formal reduction system. It reconstructs the rule set in a manner that provides the best rated solution for the entire test case set.