Extracting Logic Programs from Artificial Neural Networks

This document is essentially divided in two parts, where different methods are presented for extracting knowledge from an aritificial neural network representing an immediate consequence operator. In the first part we investigate the relationship between neurosymbolic integration (in particular the extraction of a logic program from a neural network) and inductive logic programming from a practical point of view. After a general introduction to the foundations of ILP, the task of extraction of a neural network is reformulated to fit the problem setting of ILP. We then practically test a variety of different programs and evaluate them. The second part of the document builds up a theoretical foundation for the special case of extracting propositional logic programs. We give algorithms for definite as well as normal propositional logic programs. Several theoretical results are presented, difficulties and possible solutions are observed.

[1]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .

[2]  Shan-Hwei Nienhuys-Cheng,et al.  Foundations of Inductive Logic Programming , 1997, Lecture Notes in Computer Science.

[3]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[4]  Max Bramer.  Logic Programming with Prolog , 2005, Springer London.

[5]  Stephen Muggleton,et al.  Machine Invention of First Order Predicates by Inverting Resolution , 1988, ML.

[6]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[7]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[8]  Krysia Broda,et al.  Symbolic knowledge extraction from trained neural networks: A sound approach , 2001, Artif. Intell..

[9]  Ehud Shapiro,et al.  Inductive Inference of Theories from Facts , 1991, Computational Logic - Essays in Honor of Alan Robinson.

[10]  Michael J. Maher.  Equivalences of Logic Programs , 1988, Foundations of Deductive Databases and Logic Programming..

[11]  Stephen Muggleton,et al.  Efficient Induction of Logic Programs , 1990, ALT.

[12]  Steffen Hölldobler,et al.  Approximating the Semantics of Logic Programs by Recurrent Neural Networks , 1999, Applied Intelligence.

[13]  Pascal Hitzler,et al.  Logic programs, iterated function systems, and recurrent radial basis function networks , 2004, J. Appl. Log..

[14]  P. Hitzler,et al.  The Integration of Connectionism and First-Order Knowledge Representation and Reasoning as a Challenge for Artificial Intelligence , 2004, ArXiv.

[15]  Stephen Muggleton,et al.  Learning from Positive Data , 1996, Inductive Logic Programming Workshop.

[16]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[17]  R. Mike Cameron-Jones,et al.  FOIL: A Midterm Report , 1993, ECML.

[18]  Steffen Hölldobler,et al.  Towards a New Massively Parallel Computational Model for Logic Programming , 1994 .