On connectionism and rule extraction

There are two major motivations for rule extraction from trained artificial neural networks. First, some of the proposed neural network architectures, like multiplayer perceptrons, are so complex that that it is difficult to understand the logic behind any decision or inference made by such a network. So from an engineering standpoint, rule extraction from such a complex network provides a way to understand and explain the logic behind any decision made by it. By the way, [11] define the rule extraction from neural networks task as follows: “Given a trained neural network and the examples used to train it, produce a concise and accurate symbolic description of the network.” So the objective of rule extraction is to provide a certain type of symbolic description of the network. A second major motivation for rule extraction is to bridge the divide between symbolic AI and connectionism; that is, to show that connectionist subsymbolic systems are just an implementation of higher-level symbolic systems. Thus rule-extraction and rule-insertion, whereby a connectionist network is created from a set of symbolic rules [16,17], provides a seamless integration between these two levels, the symbolic and the subsymbolic.

[1]  Patrick K. Simpson,et al.  Fuzzy min-max neural networks. I. Classification , 1992, IEEE Trans. Neural Networks.

[2]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[3]  C. Lee Giles,et al.  Extraction, Insertion and Refinement of Symbolic Rules in Dynamically Driven Recurrent Neural Networks , 1993 .

[4]  Vasant Honavar,et al.  Integrating Symbol Processing Systems and Connectionist Networks , 1995 .

[5]  James M. Keller,et al.  Incorporating Fuzzy Membership Functions into the Perceptron Algorithm , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Asim Roy.  Brain's internal mechanisms - a new paradigm , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[7]  Stephen Grossberg,et al.  Studies of mind and brain , 1982 .

[8]  Bruno Apolloni,et al.  From synapses to rules , 2002, Cognitive Systems Research.

[9]  Michael C. Mozer,et al.  Template-Based Algorithms for Connectionist Rule Extraction , 1994, NIPS.

[10]  James L. McClelland.  Putting Knowledge in its Place: A Scheme for Programming Parallel Processing Structures on the Fly , 1988, Cogn. Sci..

[11]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[12]  Stephen Grossberg,et al.  Competitive Learning: From Interactive Activation to Adaptive Resonance , 1987, Cogn. Sci..

[13]  Asim Roy,et al.  A neural-network learning theory and a polynomial time RBF algorithm , 1997, IEEE Trans. Neural Networks.

[14]  R. Shadmehr,et al.  Neural correlates of motor memory consolidation. , 1997, Science.

[15]  Kevin N. Gurney,et al.  An introduction to neural networks , 2018 .

[16]  Chin-Teng Lin,et al.  Supervised and unsupervised learning with fuzzy similarity for neural-network-based fuzzy logic control systems , 1992, [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics.

[17]  Shigeo Abe,et al.  Fuzzy rules extraction directly from numerical data for function approximation , 1995, IEEE Trans. Syst. Man Cybern..

[18]  Paul Smolensky,et al.  Connectionist modeling: Neural computation/mental connections. , 1989 .

[19]  Ron Sun,et al.  Integrating rules and connectionism for robust commonsense reasoning , 1994, Sixth-generation computer technology series.

[20]  Stephen Grossberg,et al.  Nonlinear neural networks: Principles, mechanisms, and architectures , 1988, Neural Networks.

[21]  Joachim Diederich,et al.  Survey and critique of techniques for extracting rules from trained artificial neural networks , 1995, Knowl. Based Syst..

[22]  Somnath Mukhopadhyay,et al.  Iterative generation of higher-order nets in polynomial time using linear programming , 1997, IEEE Trans. Neural Networks.

[23]  P. K. Simpson.  Fuzzy Min-Max Neural Networks-Part 1 : Classification , 1992 .

[24]  Lotfi A. Zadeh,et al.  Outline of a New Approach to the Analysis of Complex Systems and Decision Processes , 1973, IEEE Trans. Syst. Man Cybern..

[25]  Lee A. Feldkamp,et al.  Training a hybrid neural-fuzzy system , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[26]  S. Grossberg.  Studies of mind and brain : neural principles of learning, perception, development, cognition, and motor control , 1982 .

[27]  Madan M. Gupta,et al.  On the principles of fuzzy neural networks , 1994 .

[28]  Jude Shavlik,et al.  THE EXTRACTION OF REFINED RULES FROM KNOWLEDGE BASED NEURAL NETWORKS , 1993 .

[29]  Vasant Honavar,et al.  Books-Received - Artificial Intelligence and Neural Networks - Steps Toward Principled Integration , 1994 .

[30]  Geoffrey E. Hinton,et al.  Connectionist Architectures for Artificial Intelligence , 1990, Computer.

[31]  Jude W. Shavlik,et al.  Using Sampling and Queries to Extract Rules from Trained Neural Networks , 1994, ICML.

[32]  Sukhdev Khebbal,et al.  Intelligent Hybrid Systems , 1994 .

[33]  Bart Kosko,et al.  Neural networks and fuzzy systems , 1998 .

[34]  Ronald R. Yager,et al.  Generalized Fuzzy and Matrix Associative Holographic Memories , 1993, J. Intell. Fuzzy Syst..

[35]  Ah-Hwee Tan,et al.  Rule Extraction: From Neural Architecture to Symbolic Representation , 1995 .

[36]  Huan Liu,et al.  Symbolic Representation of Neural Networks , 1996, Computer.

[37]  Karl J. Friston,et al.  Neural modeling and functional brain imaging: an overview , 2000, Neural Networks.

[38]  Somnath Mukhopadhyay,et al.  A polynomial time algorithm for the construction and training of a class of multilayer perceptrons , 1993, Neural Networks.

[39]  Edward T. Lee,et al.  Fuzzy Neural Networks , 1975 .

[40]  Sankar K. Pal,et al.  Multilayer perceptron, fuzzy sets, and classification , 1992, IEEE Trans. Neural Networks.

[41]  Ronald R. Yager,et al.  Fuzzy sets, neural networks, and soft computing , 1994 .

[42]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[43]  LiMin Fu,et al.  Rule Generation from Neural Networks , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[44]  Helmut Prendinger,et al.  Approximate Reasoning , 1997, EPIA.

[45]  Huan Liu,et al.  NeuroLinear: From neural networks to oblique decision rules , 1997, Neurocomputing.

[46]  David E. Rumelhart,et al.  The architecture of mind: a connectionist approach , 1989 .

[47]  Giovanni Soda,et al.  Unified Integration of Explicit Knowledge and Learning by Example in Recurrent Networks , 1995, IEEE Trans. Knowl. Data Eng..

[48]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[49]  Asim Roy,et al.  On connectionism, rule extraction, and brain-like learning , 2000, IEEE Trans. Fuzzy Syst..

[50]  Teuvo Kohonen,et al.  An introduction to neural computing , 1988, Neural Networks.

[51]  Jerome A. Feldman,et al.  Connectionist Models and Their Properties , 1982, Cogn. Sci..

[52]  Isao Hayashi,et al.  NN-driven fuzzy reasoning , 1991, Int. J. Approx. Reason..

[53]  Asim Roy,et al.  An algorithm to generate radial basis function (RBF)-like nets for classification problems , 1995, Neural Networks.

[54]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.