The Extraction of Reened Rules from Knowledge-based Neural Networks

Neural networks, despite their empirically-proven abilities, have been little used for the renement of existing knowledge because this task requires a three-step process. First, knowledge in some form must be inserted into a neural network. Second, the network must be re ned. Third, knowledge must be extracted from the network. We have previously described a method for the rst step of this process. Standard neural learning techniques can accomplish the second step. In this paper, we propose and empirically evaluate a method for the nal, and possibly most di cult, step. This method e ciently extracts symbolic rules from trained neural networks. The four major results of empirical tests of this method are that the extracted rules: (1) closely reproduce (and can even exceed) the accuracy of the network from which they are extracted; (2) are superior to the rules produced by methods that directly re ne symbolic rules; (3) are superior to those produced by previous techniques for extracting rules from trained neural networks; (4) are \human comprehensible." Thus, the method demonstrates that neural networks can be an e ective tool for the re nement of symbolic knowledge. Moreover, the rule-extraction technique developed herein contributes to the understanding of how symbolic and connectionist approaches to arti cial intelligence can be pro tably integrated.