Experimental comparison of recursive self-organizing maps for processing tree-structured data

Self-organizing neural network models have recently been extended to more general data structures, such as sequences or trees. We empirically compare three recursive models of the self-organizing map-SOMSD, MSOM and RecSOM-using three different tree data sets with the increasing level of complexity: binary syntactic trees, ternary linguistic propositions and 5-ary graphical data. We evaluate the models in terms of our four proposed measures focusing on unit's memory depth and on capability to differentiate among the trees, and using two additional measures introduced earlier, focusing on statistics of labels distribution and the spatiotemporal information encoded into the maps. We focus on models' ability to differentiate among the trees and find out that the models differ in how they balance the effects of leaves and the tree structure in achieving this task. SOMSD turns out to have the highest sensitivity to structural information that affects the tree clustering. With respect to input differentiation, only RecSOM can assign unique output representations to different inputs in all three data sets. We also argue that MSOM, despite the commutativity of operation (with respect to children) for calculating context representation, has the potential to differentiate the binary trees with permuted children.

[1]  T. Kohonen,et al.  Bibliography of Self-Organizing Map SOM) Papers: 1998-2001 Addendum , 2003 .

[2]  Alessandro Sperduti,et al.  A general framework for adaptive processing of data structures , 1998, IEEE Trans. Neural Networks.

[3]  Michael M. Richter,et al.  Adaptivity and Learning , 2003 .

[4]  Barbara Hammer,et al.  Perspectives on learning symbolic data with connectionistic systems , 2003 .

[5]  Tommi S. Jaakkola,et al.  Fast optimal leaf ordering for hierarchical clustering , 2001, ISMB.

[6]  Barbara Hammer,et al.  Neural methods for non-standard data , 2004, ESANN.

[7]  Ah Chung Tsoi,et al.  A self-organizing map for adaptive processing of structured data , 2003, IEEE Trans. Neural Networks.

[8]  T. Kohonen,et al.  Self-organizing semantic maps , 1989, Biological Cybernetics.

[9]  Thomas Voegtlin,et al.  Recursive self-organizing maps , 2002, Neural Networks.

[10]  Jukka Heikkonen,et al.  Time Series Predicition using Recurrent SOM with Local Linear Models , 1997 .

[11]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[12]  Aluizio F. R. Araújo,et al.  A Taxonomy for Spatiotemporal Connectionist Networks Revisited: The Unsupervised Case , 2003, Neural Computation.

[13]  Marco Gori,et al.  Terminal attractor algorithms: A critical analysis , 1997, Neurocomputing.

[14]  Alessandro Sperduti,et al.  Unsupervised clustering of continuous trajectories of kinematic trees with SOM-SD , 2006, ESANN.

[15]  Peter Tiño,et al.  Dynamics and Topographic Organization of Recursive Self-Organizing Maps , 2006, Neural Computation.

[16]  John F. Kolen,et al.  From Sequences to Data Structures: Theory and Applications , 2001 .

[17]  Ah Chung Tsoi,et al.  Special issue on recurrent neural networks for sequence processing , 1997 .

[18]  Alessandro Sperduti,et al.  Supervised neural networks for the classification of structures , 1997, IEEE Trans. Neural Networks.

[19]  Samuel Kaski,et al.  Bibliography of Self-Organizing Map (SOM) Papers: 1981-1997 , 1998 .

[20]  Barbara Hammer,et al.  Merge SOM for temporal data , 2005, Neurocomputing.

[21]  John G. Taylor,et al.  The temporal Kohönen map , 1993, Neural Networks.

[22]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[23]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[24]  Igor Farkaš,et al.  INVESTIGATING SYSTEMATICITY IN THE LINEAR RAAM NEURAL NETWORK , 2009 .

[25]  Alessio Micheli,et al.  Recursive self-organizing network models , 2004, Neural Networks.

[26]  Alessio Micheli,et al.  A general framework for unsupervised processing of structured data , 2004, Neurocomputing.

[27]  Tong Tang,et al.  Proceedings of the European Symposium on Artificial Neural Networks , 2006 .

[28]  Barbara Hammer,et al.  Learning with recurrent neural networks , 2000 .