Neural Networks for Processing Data Structures

The possibility to represent and to process structures in a neural network greatly increases the computational capability of neural networks. This new capability, besides to provide a new tool for the classification of structures, can also be exploited to integrate neural networks and symbolic systems in a hybrid system. In fact, structures generated by a symbolic module can be evaluated by this type of networks and their evaluation can be used to modify the behavior of the symbolic module. An instance of this integration scheme is given, for example, by learning heuristics for automated deduction systems. Goller reported very successful results in using a Back-propagation Through Structure network within the SETHEO theorem prover [8]. On the other side, it is not difficult to figure out, in analogy with finite state automata extraction from recurrent networks, how to extract tree automata from a neural network for structures. This would allow the above scheme to work on the other side around, with a neural module which is driven by a symbolic subsystem.

[1]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[2]  Stefan C. Kremer,et al.  Finite State Automata that Recurrent Cascade-Correlation Cannot Represent , 1995, NIPS.

[3]  Srimat T. Chakradhar,et al.  First-order versus second-order single-layer recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[4]  Alessandro Sperduti,et al.  Supervised neural networks for the classification of structures , 1997, IEEE Trans. Neural Networks.

[5]  Michael G. Thomason,et al.  Syntactic Pattern Recognition, An Introduction , 1978, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Christoph Goller,et al.  Inductive Learning in Symbolic Domains Using Structure-Driven Recurrent Neural Networks , 1996, KI.

[7]  A. Jennings,et al.  Structurally adaptive self-organizing neural trees , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[8]  C. Lee Giles,et al.  Constructive learning of recurrent neural networks: limitations of recurrent cascade correlation and a simple solution , 1995, IEEE Trans. Neural Networks.

[9]  Don R. Hush,et al.  Bounds on the complexity of recurrent neural network implementations of finite state machines , 1993, Neural Networks.

[10]  Ronald A. Cole,et al.  A performance comparison of trained multilayer perceptrons and trained classification trees , 1990 .

[11]  Wayne Ieee,et al.  Entropy Nets: From Decision Trees to Neural Networks , 1990 .

[12]  P. Schulz.  Some Experiments on the Applicability of Folding Architecture Networks to Guide Theorem Proving , 1997 .

[13]  Noga Alon,et al.  Efficient simulation of finite automata by neural nets , 1991, JACM.

[14]  Stefan C. Kremer,et al.  Comments on "Constructive learning of recurrent neural networks: limitations of recurrent cascade correlation and a simple solution" , 1996, IEEE Trans. Neural Networks.

[15]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[16]  M. Goudreau,et al.  First-order vs. Second-order Single Layer Recurrent Neural Networks , 1994 .

[17]  C. Lee Giles,et al.  Constructing deterministic finite-state automata in recurrent neural networks , 1996, JACM.

[18]  R. Mammone,et al.  Neural tree networks , 1992 .

[19]  Jean-Pierre Nadal,et al.  Neural trees: a new tool for classification , 1990 .

[20]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[21]  Alessandro Sperduti,et al.  Learning Distributed Representations for the Classification of Terms , 1995, IJCAI.

[22]  M. P. Perrone,et al.  A soft-competitive splitting rule for adaptive tree-structured neural networks , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[23]  Scott E. Fahlman,et al.  The Recurrent Cascade-Correlation Architecture , 1990, NIPS.

[24]  T. Kailath,et al.  Discrete Neural Computation: A Theoretical Foundation , 1995 .

[25]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[26]  Nathan Intrator,et al.  Unsupervised splitting rules for neural tree classifiers , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[27]  Barbara Hammer,et al.  Neural networks can approximate mappings on structured objects , 1997 .

[28]  Christoph Goller,et al.  Learning task-dependent distributed representations by backpropagation through structure , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[29]  Hava T. Siegelmann,et al.  On the Computational Power of Neural Nets , 1995, J. Comput. Syst. Sci..

[30]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[31]  Christoph Goller,et al.  A connectionist approach for learning search-control heuristics for automated deduction systems , 1999, DISKI.

[32]  james w.thatcher,et al.  tree automata: an informal survey , 1974 .

[33]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .