The Role of Test Cases in Automated Knowledge Refinement

This paper considers test case selection strategies for the automated knowledge refinement of rule-based expert systems. Refinement systems rely heavily on test cases to find faults in a KB, but also to generate and validate repairs. We argue that test cases should not be regarded as a fixed input to the refinement process, but that they should be actively chosen in order to discriminate between alternative refinements. Further, since test cases may be difficult or expensive to obtain, it is desirable to minimise the number required during refinement. A selection strategy is proposed which identifies test cases that will attempt to refute the suitability of each proposed refinement. This approach significantly improves the quality of the refined KBs, while also reducing the number of cases required.