Rule Extraction: From Neural Architecture to Symbolic Representation

This paper shows how knowledge, in the form of fuzzy rules, can be derived from a supervised learning neural network called fuzzy ARTMAP. Rule extraction proceeds in two stages: pruning, which simplifies the network structure by removing excessive recognition categories and weights; and quantization of continuous learned weights, which allows the final system state to be translated into a usable set of descriptive rules. Three benchmark studies illustrate the rule extraction methods: (1) Pima Indian diabetes diagnosis, (2) mushroom classification and (3) DNA promoter recognition. Fuzzy ARTMAP and ART-EMAP are compared with the ADAP algorithm, the k nearest neighbor system, the back-propagation network and the C4.5 decision tree. The ARTMAP rule extraction procedure is also compared with the Knowledgetron and NOFM algorithms, which extract rules from back-propagation networks. Simulation results consistently indicate that ARTMAP rule extraction produces compact sets of comprehensible rules for which accura...

[1]  Limin Fu.  A neural network model for learning rule-based systems , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[2]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[3]  Gail A. Carpenter,et al.  ART-EMAP: A neural network architecture for object recognition by evidence accumulation , 1995, IEEE Trans. Neural Networks.

[4]  Jude W. Shavlik,et al.  Learning Symbolic Rules Using Artificial Neural Networks , 1993, ICML.

[5]  Stephen Grossberg,et al.  Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks , 1973 .

[6]  Steven L. Salzberg,et al.  Learning with Nested Generalized Exemplars , 1990 .

[7]  Geoffrey E. Hinton,et al.  Simplifying Neural Networks by Soft Weight-Sharing , 1992, Neural Computation.

[8]  Stephen Grossberg,et al.  ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[9]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .

[10]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[11]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[12]  Jude W. Shavlik,et al.  Interpretation of Artificial Neural Networks: Mapping Knowledge-Based Neural Networks into Rules , 1991, NIPS.

[13]  J. C. Schlimmer,et al.  Concept acquisition through representational adjustment , 1987 .

[14]  Ah-Hwee Tan.  Synthesizing neural network and symbolic knowledge processing , 1994 .

[15]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[16]  Richard S. Johannes,et al.  Using the ADAP Learning Algorithm to Forecast the Onset of Diabetes Mellitus , 1988 .

[17]  Bart Kosko,et al.  Fuzzy entropy and conditioning , 1986, Inf. Sci..

[18]  S. Grossberg,et al.  Pattern Recognition by Self-Organizing Neural Networks , 1991 .

[19]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..