Symbolic knowledge and neural networks: insertion, refinement and extraction

Explanation-based and empirical learning are two largely complementary methods of machine learning. These approaches to machine learning both have serious problems which preclude their being a general purpose learning method. However, a "hybrid" learning method that combines explanation-based with empirical learning may be able to use the strengths of one learning method to address the weaknesses of the other method. Hence, a system that effectively combines the two approaches to learning can be expected to be superior to either approach in isolation. This thesis describes a hybrid system called K scBANN which is shown to be an effective combination of these two learning methods. K scBANN (Knowledge-Based Artificial Neural Networks) is a three-part hybrid learning system built on top of "neural" learning techniques. The first part uses a set of approximately-correct rules to determine the structure and initial link weights of an artificial neural network, thereby making the rules accessible for modification by neural learning. The second part of K scBANN modifies the resulting network using essentially standard neural learning techniques. The third part of K scBANN extracts refined rules from trained networks. K scBANN is evaluated by empirical tests in the domain of molecular biology. Networks created by K scBANN are shown to be superior, in terms of their ability to correctly classify unseen examples, to a wide variety of learning systems as well as techniques proposed by experts in the problems investigated. In addition, empirical tests show that K scBANN is robust to errors in the initial rules and insensitive to problems resulting from the presence of extraneous input features. The third part of K scBANN, which extracts rules from trained networks, addresses a significant problem in the use of neural networks--understanding what a neural network learns. Empirical tests of the proposed rule-extraction method show that it simplifies understanding of trained networks by reducing the number of: consequents (hidden units), antecedents (weighted links), and possible antecedent weights. Surprisingly, the extracted rules are often more accurate at classifying examples not seen during training than the trained network from which they came.