Extracting rules from neural networks using symbolic algorithms: preliminary results

Although Artificial Neural Networks (ANNs) have been satisfactorily employed in several problems, such as clustering, pattern recognition, dynamic systems control and prediction, they still suffer from significant limitations. One of them is that the induced concept representation is not usually comprehensible to humans. Several techniques have been suggested to extract meaningful knowledge from trained ANNs. This paper proposes the use of symbolic learning algorithms, commonly used by the Machine Learning community, to extract symbolic representations from trained ANNs. The procedure proposed is similar to that used by the Trepan algorithm (Craven, 1996), which extracts comprehensible, symbolic representations (decision trees) from trained ANNs.

[1]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[2]  R. Nakano,et al.  Medical diagnostic expert system based on PDP model , 1988, IEEE 1988 International Conference on Neural Networks.

[3]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[4]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[5]  LiMin Fu,et al.  Rule Generation from Neural Networks , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[6]  Jean-Gabriel Ganascia,et al.  A Bayesian Framework to Integrate Symbolic and Neural Learning , 1994, ICML.

[7]  Jude Shavlik,et al.  THE EXTRACTION OF REFINED RULES FROM KNOWLEDGE BASED NEURAL NETWORKS , 1993 .

[8]  LiMin Fu,et al.  Rule Learning by Searching on Adapted Nets , 1991, AAAI.

[9]  Juan Moreno García,et al.  A neural architecture for the identification of number sequences , 1998, Proceedings 5th Brazilian Symposium on Neural Networks (Cat. No.98EX209).

[10]  Yoichi Hayashi,et al.  A Neural Expert System with Automated Extraction of Fuzzy If-Then Rules , 1990, NIPS.

[11]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[12]  L. F. Pau,et al.  Explanation facility for neural networks , 1992 .

[13]  Renu Vig,et al.  Medical Diagnostic Expert System , 1993, HCI.

[14]  Curtis Hall.  Neural net technology: Ready for prime time? , 1992, IEEE Expert.

[15]  Joachim Diederich,et al.  Knowledge Extraction and Recurrent Neural Networks: An Analysis of an Elman Network trained on a Natural Language Learning Task , 1998, CoNLL.

[16]  Chris Aldrich,et al.  ANN-DT: an algorithm for extraction of decision trees from artificial neural networks , 1999, IEEE Trans. Neural Networks.

[17]  Mark Craven,et al.  Extracting comprehensible models from trained neural networks , 1996 .

[18]  Jessica Lowell.  Neural Network , 2001 .

[19]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[20]  Joachim Diederich,et al.  The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks , 1998, IEEE Trans. Neural Networks.

[21]  Jude W. Shavlik,et al.  in Advances in Neural Information Processing , 1996 .

[22]  Sebastian Thrun,et al.  Extracting Rules from Artifical Neural Networks with Distributed Representations , 1994, NIPS.