A general framework for symbol and rule extraction in neural networks

We split the rule extraction task into a subsymbolic and a symbolic phase and present a set of neural networks for filling the former. Under the two general commitments of: (i) having a learning algorithm that is sensitive to feedback signals coming from the latter phase, and (ii) extracting Boolean variables whose meaning is determined by the further symbolic processing, we introduce three unsupervised learning algorithms and show related numerical examples for a multilayer perceptron, recurrent neural networks, and a specially devised vector quantizer.

[1]  John Haugeland.  The nature and plausibility of Cognitivism , 1978, Behavioral and Brain Sciences.

[2]  Bruno Apolloni,et al.  Constructing symbols as manipulable structures by recurrent networks , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[3]  Bruno Apolloni,et al.  PAC Learning of Concept Classes Through the Boundaries of Their Items , 1997, Theor. Comput. Sci..

[4]  John G. Taylor,et al.  Storing temporal sequences , 1991, Neural Networks.

[5]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[6]  Ron Sun,et al.  Integrating rules and connectionism for robust commonsense reasoning , 1994, Sixth-generation computer technology series.

[7]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[8]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[9]  C. Lee Giles,et al.  Extraction of rules from discrete-time recurrent neural networks , 1996, Neural Networks.

[10]  Lai-Wan Chan,et al.  How to Design a Connectionist Holistic Parser , 1999, Neural Computation.

[11]  John G. Taylor,et al.  Learning to generate temporal sequences by models of frontal lobes , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[12]  Joachim Diederich,et al.  The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks , 1998, IEEE Trans. Neural Networks.

[13]  Bruno Apolloni,et al.  Learning fuzzy decision trees , 1998, Neural Networks.

[14]  David Barber,et al.  Generative vector quantisation , 1999 .

[15]  Bruno Apolloni,et al.  Gaining degrees of freedom in subsymbolic learning , 2001, Theor. Comput. Sci..

[16]  John G. Taylor,et al.  The temporal Kohönen map , 1993, Neural Networks.