Sequencing Training Examples for Iterative Knowledge Refinement

Refinement tools seek to correct faulty knowledge based systems (KBSs) by identifying and repairing faults that are indicated by training examples for which the KBS gives an incorrect solution. Refinement tools typically use a hill-climbing search to identify suitable repairs. Backtracking search algorithms, developed for constraint satisfaction problems, have been incorporated with an iterative knowledge refinement tool, to solve local maxima problems. This paper investigates how the efficiency of such a tool can be improved and introduces new and general heuristics for ordering training examples. Experimental results reveal that these heuristics applied to static and dynamic ordering of training examples can significantly improve the efficiency of the iterative refinement tool, without increasing the error-rate of the final refined KBS.

[1]  Rina Dechter,et al.  Look-Ahead Value Ordering for Constraint Satisfaction Problems , 1995, IJCAI.

[2]  Edward M. Reingold,et al.  Backtrack programming techniques , 1975, CACM.

[3]  Norman M. Sadeh,et al.  Variable and Value Ordering Heuristics for the Job Shop Scheduling Constraint Satisfaction Problem , 1996, Artif. Intell..

[4]  Peter van Beek,et al.  A Theoretical Evaluation of Selected Backtracking Algorithms , 1995, IJCAI.

[5]  Susan Craw,et al.  Incorporating Backtracking Search with Knowledge Refinement , 1999, EUROVAV.

[6]  Susan Craw,et al.  Representing Problem-Solving for Knowledge Refinement , 1999, AAAI/IAAI.

[7]  Edward P. K. Tsang,et al.  Foundations of constraint satisfaction , 1993, Computation in cognitive science.

[8]  Rina Dechter,et al.  Experimental Evaluation of Preprocessing Algorithms for Constraint Satisfaction Problems , 1994, Artif. Intell..

[9]  Barbara M. Smith,et al.  Trying Harder to Fail First , 1998, ECAI.

[10]  J. Gaschnig.  Performance measurement and analysis of certain search algorithms. , 1979 .

[11]  Raymond J. Mooney,et al.  Theory Refinement Combining Analytical and Empirical Methods , 1994, Artif. Intell..

[12]  Susan Craw,et al.  Knowledge Refinement for a Design System , 1997, EKAW.

[13]  Toby Walsh,et al.  An Empirical Study of Dynamic Variable Ordering Heuristics for the Constraint Satisfaction Problem , 1996, CP.

[14]  Steven Minton,et al.  Minimizing Conflicts: A Heuristic Repair Method for Constraint Satisfaction and Scheduling Problems , 1992, Artif. Intell..

[15]  Robert M. Haralick,et al.  Increasing Tree Search Efficiency for Constraint Satisfaction Problems , 1979, Artif. Intell..

[16]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .