Dynamic topology learning with the probabilistic self-organizing graph

Abstract Self-organizing neural networks are usually focused on prototype learning, while the topology is held fixed during the learning process. Here a method to adapt the topology of the network so that it reflects the internal structure of the input distribution is proposed. This leads to a self-organizing graph, where each unit is a mixture component of a mixture of Gaussians (MoG). The corresponding update equations are derived from the stochastic approximation framework. This approach combines the advantages of probabilistic mixtures with those of self-organization. Experimental results are presented to show the self-organization ability of our proposal and its performance when used with multivariate datasets in classification and image segmentation tasks.

[1]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[2]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[3]  Hau-San Wong,et al.  3D head model retrieval in kernel feature space using HSOM , 2008, Pattern Recognit..

[4]  Aluizio F. R. Araújo,et al.  Local adaptive receptive field self-organizing map for image color segmentation , 2009, Image Vis. Comput..

[5]  Domingo López-Rodríguez,et al.  Probabilistic PCA Self-Organizing Maps , 2009, IEEE Transactions on Neural Networks.

[6]  Ravi Kothari,et al.  Visualization and Analysis of System Monitoring Data using Multi-resolution Context Information , 2008, IEEE Transactions on Network and Service Management.

[7]  Hsin-Min Wang,et al.  Model-Based Clustering by Probabilistic Self-Organizing Maps , 2009, IEEE Transactions on Neural Networks.

[8]  Constantine Kotropoulos,et al.  Assessment of self-organizing map variants for clustering with application to redistribution of emotional speech patterns , 2007, Neurocomputing.

[9]  Helge J. Ritter,et al.  Detecting, Assessing and Monitoring Relevant Topics in Virtual Information Environments , 2009, IEEE Transactions on Knowledge and Data Engineering.

[10]  Chaur-Heh Hsieh,et al.  Classified self-organizing map with adaptive subcodebook for edge preserving vector quantization , 2009, Neurocomputing.

[11]  Satoru Kawai,et al.  An Algorithm for Drawing General Undirected Graphs , 1989, Inf. Process. Lett..

[12]  Davide Bacciu,et al.  Competitive Repetition Suppression (CoRe) Clustering: A Biologically Inspired Learning Model With Application to Robust Clustering , 2008, IEEE Transactions on Neural Networks.

[13]  Xiaolong Wang,et al.  ConSOM: A conceptional self-organizing map model for text clustering , 2008, Neurocomputing.

[14]  Shen Furao,et al.  An enhanced self-organizing incremental neural network for online unsupervised learning , 2007, Neural Networks.

[15]  Toru Fujinaka,et al.  Automatic Inspection of Transmission Devices Using Acoustic Data , 2008, IEEE Transactions on Automation Science and Engineering.

[16]  Shen Furao,et al.  An incremental network for on-line unsupervised classification and topology learning , 2006, Neural Networks.

[17]  Ezequiel López-Rubio.  Probabilistic Self-Organizing Maps for Continuous Data , 2010, IEEE Transactions on Neural Networks.

[18]  Marc M. Van Hulle.  Joint Entropy Maximization in Kernel-Based Topographic Maps , 2002, Neural Computation.

[19]  Heba M. Lakany,et al.  Extracting a diagnostic gait signature , 2008, Pattern Recognit..

[20]  Ezequiel López-Rubio,et al.  Multivariate Student-t self-organizing maps , 2009, Neural Networks.

[21]  Chip-Hong Chang,et al.  New adaptive color quantization method based on self-organizing maps , 2005, IEEE Transactions on Neural Networks.

[22]  Tommy W. S. Chow,et al.  PolSOM: A new method for multidimensional data visualization , 2010, Pattern Recognit..

[23]  Tom Heskes,et al.  Self-organizing maps, vector quantization, and mixture modeling , 2001, IEEE Trans. Neural Networks.

[24]  Duane DeSieno,et al.  Adding a conscience to competitive learning , 1988, IEEE 1988 International Conference on Neural Networks.

[25]  Ming Xie,et al.  Color clustering and learning for image segmentation based on neural networks , 2005, IEEE Trans. Neural Networks.

[26]  Kunihiko Fukushima,et al.  Self-organization of shift-invariant receptive fields , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[27]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[28]  Riccardo Rizzo,et al.  A comparison between habituation and conscience mechanism in self-organizing maps , 2006, IEEE Transactions on Neural Networks.

[29]  Ezequiel López-Rubio,et al.  Self-Organizing Dynamic Graphs , 2004, Neural Processing Letters.

[30]  Alfredo Vellido,et al.  Missing data imputation through GTM as a mixture of t-distributions , 2006, Neural Networks.

[31]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[32]  Ben J. A. Kröse,et al.  Self-organizing mixture models , 2005, Neurocomputing.

[33]  Zhang Yi,et al.  Shape recovery by a generalized topology preserving SOM , 2008, Neurocomputing.

[34]  Emden R. Gansner,et al.  An open graph visualization system and its applications to software engineering , 2000 .

[35]  Sim Heng Ong,et al.  Colour image segmentation using the self-organizing map and adaptive resonance theory , 2005, Image Vis. Comput..

[36]  Bunyarit Uyyanonvara,et al.  Novel fast color reduction algorithm for time-constrained applications , 2005, J. Vis. Commun. Image Represent..

[37]  Hujun Yin,et al.  ViSOM - a novel method for multivariate data projection and structure visualization , 2002, IEEE Trans. Neural Networks.

[38]  Shin Ishii,et al.  On-line EM Algorithm for the Normalized Gaussian Network , 2000, Neural Computation.

[39]  Aluizio F. R. Araújo,et al.  Growing Self-Organizing Surface Map: Learning a Surface Topology from a Point Cloud , 2010, Neural Computation.

[40]  Chaur-Heh Hsieh,et al.  Sample-size adaptive self-organization map for color images quantization , 2007, Pattern Recognit. Lett..

[41]  Peter Tiño,et al.  Visualization of Tree-Structured Data Through Generative Topographic Mapping , 2008, IEEE Transactions on Neural Networks.

[42]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .