Symbolic knowledge extraction from trained neural networks: A sound approach

Although neural networks have shown very good performance in many application domains, one of their main drawbacks lies in the incapacity to provide an explanation for the underlying reasoning mechanisms. The “explanation capability” of neural networks can be achieved by the extraction of symbolic knowledge. In this paper, we present a new method of extraction that captures nonmonotonic rules encoded in the network, and prove that such a method is sound. We start by discussing some of the main problems of knowledge extraction methods. We then discuss how these problems may be ameliorated. To this end, a partial ordering on the set of input vectors of a network is defined, as well as a number of pruning and simplification rules. The pruning rules are then used to reduce the search space of the extraction algorithm during a pedagogical extraction, whereas the simplification rules are used to reduce the size of the extracted set of rules. We show that, in the case of regular networks, the extraction algorithm is sound and complete. We proceed to extend the extraction algorithm to the class of non-regular networks, the general case. We show that non-regular networks always contain regularities in their subnetworks. As a result, the underlying extraction method for regular networks can be applied, but now in a decompositional fashion. In order to combine the sets of rules extracted from each subnetwork into the final set of rules, we use a method whereby we are able to keep the soundness of the extraction algorithm. Finally, we present the results of an empirical analysis of the extraction system, using traditional examples and real-world application problems. The results have shown that a very high fidelity between the extracted set of rules and the network can be achieved.

[1]  Mark Craven,et al.  Extracting comprehensible models from trained neural networks , 1996 .

[2]  Sebastian Thrun,et al.  Extracting Provably Correct Rules from Artificial Neural Networks , 1993 .

[3]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[4]  Sebastian Thrun,et al.  The MONK''s Problems-A Performance Comparison of Different Learning Algorithms, CMU-CS-91-197, Sch , 1991 .

[5]  Jacek M. Zurada,et al.  Knowledge-based neurocomputing , 2000 .

[6]  Geoffrey G. Towell,et al.  Symbolic knowledge and neural networks: insertion, refinement and extraction , 1992 .

[7]  Victor W. Marek,et al.  Nonmonotonic logic - context-dependent reasoning , 1997, Artificial intelligence.

[8]  Henry Prakken,et al.  Argument-Based Extended Logic Programming with Defeasible Priorities , 1997, J. Appl. Non Class. Logics.

[9]  J. Lloyd.  Foundations of Logic Programming , 1984, Symbolic Computation.

[10]  Gerson Zaverucha,et al.  An integration of neural networks and nonmonotonic reasoning for power system diagnosis , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[11]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[12]  Marvin Minsky,et al.  Logical Versus Analogical or Symbolic Versus Connectionist or Neat Versus Scruffy , 1991, AI Mag..

[13]  Jacek M. Zurada,et al.  Extraction of linguistic rules from data via neural networks and fuzzy approximation , 2000 .

[14]  Bart Selman,et al.  Horn Approximations of Empirical Data , 1995, Artif. Intell..

[15]  Michael C. Mozer,et al.  Template-Based Algorithms for Connectionist Rule Extraction , 1994, NIPS.

[16]  Dov M. Gabbay,et al.  Handbook of logic in artificial intelligence and logic programming (vol. 1) , 1993 .

[17]  Steffen Hölldobler,et al.  Towards a New Massively Parallel Computational Model for Logic Programming , 1994 .

[18]  Hector J. Levesque,et al.  A New Method for Solving Hard Satisfiability Problems , 1992, AAAI.

[19]  Jude Shavlik,et al.  THE EXTRACTION OF REFINED RULES FROM KNOWLEDGE BASED NEURAL NETWORKS , 1993 .

[20]  Artur S. d'Avila Garcez,et al.  Applying the connectionist inductive learning and logic programming system to power system diagnosis , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[21]  Rudy Setiono,et al.  Extracting Rules from Neural Networks by Pruning and Hidden-Unit Splitting , 1997, Neural Computation.

[22]  F. Maire,et al.  A partial order for the M-of-N rule-extraction algorithm , 1997, IEEE Trans. Neural Networks.

[23]  N. K. Bose,et al.  Neural Network Fundamentals with Graphs, Algorithms and Applications , 1995 .

[24]  Rudy Setiono,et al.  A Penalty-Function Approach for Pruning Feedforward Neural Networks , 1997, Neural Computation.

[25]  Saso Dzeroski,et al.  Inductive Logic Programming: Techniques and Applications , 1993 .

[26]  Grace Jordison.  Molecular Biology of the Gene , 1965, The Yale Journal of Biology and Medicine.

[27]  Donald Nute,et al.  Defeasible Logic , 1994, INAP.

[28]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[29]  LiMin Fu,et al.  Neural networks in computer intelligence , 1994 .

[30]  Melanie Hilario,et al.  An Overview of Strategies for Neurosymbolic Integration , 1995 .

[31]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[32]  Jude W. Shavlik,et al.  Using Sampling and Queries to Extract Rules from Trained Neural Networks , 1994, ICML.

[33]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[34]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..