Processing directed acyclic graphs with recursive neural networks

Recursive neural networks are conceived for processing graphs and extend the well-known recurrent model for processing sequences. In Frasconi et al. (1998), recursive neural networks can deal only with directed ordered acyclic graphs (DOAGs), in which the children of any given node are ordered. While this assumption is reasonable in some applications, it introduces unnecessary constraints in others. In this paper, it is shown that the constraint on the ordering can be relaxed by using an appropriate weight sharing, that guarantees the independence of the network output with respect to the permutations of the arcs leaving from each node. The method can be used with graphs having low connectivity and, in particular, few outcoming arcs. Some theoretical properties of the proposed architecture are given. They guarantee that the approximation capabilities are maintained, despite the weight sharing.

[1]  Franco Scarselli,et al.  Learning User Profiles in NAUTILUS , 2000, AH.

[2]  Barbara Hammer,et al.  Learning with recurrent neural networks , 2000 .

[3]  Andreas Küchler,et al.  Adaptive processing of structural data: from sequences to trees and beyond , 2000 .

[4]  Alessio Micheli,et al.  Analysis of the Internal Representations Developed by Neural Networks for Structures Applied to Quantitative Structure-Activity Relationship Studies of Benzodiazepines , 2001, J. Chem. Inf. Comput. Sci..

[5]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[6]  Marco Gori,et al.  Theoretical properties of recursive neural networks with linear neurons , 2001, IEEE Trans. Neural Networks.

[7]  Barbara Hammer,et al.  Approximation capabilities of folding networks , 1999, ESANN.

[8]  Christoph Goller,et al.  Inductive Learning in Symbolic Domains Using Structure-Driven Recurrent Neural Networks , 1996, KI.

[9]  Reiner Lenz,et al.  Optimal filters for the detection of linear patterns in 2-D and higher dimensional images , 1987, Pattern Recognit..

[10]  Alessandro Sperduti,et al.  A general framework for adaptive processing of data structures , 1998, IEEE Trans. Neural Networks.

[11]  C. Curtis,et al.  Representation theory of finite groups and associated algebras , 1962 .

[12]  W. Schempp,et al.  Harmonic analysis on the Heisenberg nilpotent Lie group, with applications to signal theory , 1986 .

[13]  Andrew Zisserman,et al.  Applications of Invariance in Computer Vision , 1993, Lecture Notes in Computer Science.

[14]  Din-Chang Tseng,et al.  Invariant handwritten Chinese character recognition using fuzzy min-max neural networks , 1997, Pattern Recognit. Lett..

[15]  Ah Chung Tsoi,et al.  Universal Approximation Using Feedforward Neural Networks: A Survey of Some Existing Methods, and Some New Results , 1998, Neural Networks.

[16]  Barbara Hammer,et al.  Neural networks can approximate mappings on structured objects , 1997 .

[17]  Eduardo D. Sontag,et al.  For neural networks, function determines form , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[18]  Alessandro Sperduti,et al.  Supervised neural networks for the classification of structures , 1997, IEEE Trans. Neural Networks.

[19]  Lars Asplund,et al.  Neural networks for admission control in an ATM network , 1994 .

[20]  Barbara Hammer,et al.  Generalization Ability of Folding Networks , 2001, IEEE Trans. Knowl. Data Eng..

[21]  Christoph Goller,et al.  A connectionist approach for learning search-control heuristics for automated deduction systems , 1999, DISKI.

[22]  Reiner Lenz.  A group theoretical approach to filter design , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[23]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[24]  Jouko Lampinen,et al.  Improved rotational invariance for statistical inverse in electrical impedance tomography , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.