Linear manifold topographic map formation based on an energy function with on-line adaptation rules

The lack of an energy function is an important problem in many topographic map formation methods. This paper describes formation of a map, called linear manifold topographic map, based on minimization of an energy function. Using multiple low-dimensional linear manifolds as data representation elements, the data distributions of many problems with high-dimensional data spaces can be simply and parsimoniously modeled. Two sets of on-line adaptation rules are obtained based on stochastic gradient descent on the energy functions devised for a soft and a hard data assignment. Experimental results show good performance of the map in comparison to other relevant techniques.

[1]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[2]  Reza Safabakhsh,et al.  Information Maximization in a Linear Manifold Topographic Map , 2009, Neural Processing Letters.

[3]  Marc M. Van Hulle,et al.  Maximum Likelihood Topographic Map Formation , 2005, Neural Computation.

[4]  Tom Heskes,et al.  Self-organizing maps, vector quantization, and mixture modeling , 2001, IEEE Trans. Neural Networks.

[5]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[6]  Hong Yan,et al.  Handwritten digit recognition by adaptive-subspace self-organizing map (ASSOM) , 1999, IEEE Trans. Neural Networks.

[7]  T. Hastie.  Principal Curves and Surfaces , 1984 .

[8]  Simon Haykin,et al.  Optimally adaptive transform coding , 1995, IEEE Trans. Image Process..

[9]  Erkki Oja,et al.  Kohonen Maps , 1999, Encyclopedia of Machine Learning.

[10]  Javier Ruiz-del-Solar,et al.  TEXSOM: Texture segmentation using self-organizing maps , 1998, Neurocomputing.

[11]  Reza Safabakhsh,et al.  Joint Entropy Maximization in the Kernel-Based Linear Manifold Topographic Map , 2007, 2007 International Joint Conference on Neural Networks.

[12]  Francesco Camastra,et al.  Data dimensionality estimation methods: a survey , 2003, Pattern Recognit..

[13]  Zhi-Qiang Liu.  Adaptive Subspace Self-Organizing Map and Its Applications in Face Recognition , 2002, Int. J. Image Graph..

[14]  E. Oja,et al.  Independent Component Analysis , 2013 .

[15]  Michael I. Jordan,et al.  On Convergence Properties of the EM Algorithm for Gaussian Mixtures , 1996, Neural Computation.

[16]  Christopher M. Bishop,et al.  GTM: A Principled Alternative to the Self-Organizing Map , 1996, NIPS.

[17]  Juan Miguel Ortiz-de-Lazcano-Lobato,et al.  Principal Components Analysis Competitive Learning , 2003, Neural Computation.

[18]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[19]  Christopher M. Bishop,et al.  Developments of the generative topographic mapping , 1998, Neurocomputing.

[20]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[21]  Geoffrey E. Hinton,et al.  Modeling the manifolds of images of handwritten digits , 1997, IEEE Trans. Neural Networks.

[22]  Ezequiel López-Rubio,et al.  A principal components analysis self-organizing map , 2004, Neural Networks.

[23]  Teuvo Kohonen,et al.  Emergence of invariant-feature detectors in the adaptive-subspace self-organizing map , 1996, Biological Cybernetics.

[24]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[25]  Michael I. Jordan,et al.  Mixtures of Probabilistic Principal Component Analyzers , 2001 .

[26]  T. Heskes.  Energy functions for self-organizing maps , 1999 .

[27]  Juha Karhunen,et al.  Principal component neural networks — Theory and applications , 1998, Pattern Analysis and Applications.

[28]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[29]  Nanda Kambhatla,et al.  Dimension Reduction by Local Principal Component Analysis , 1997, Neural Computation.

[30]  Padraig Cunningham,et al.  Adaptive Offset Subspace Self-Organizing Map: An Application to Handwritten Digit Recognition , 2006 .

[31]  Edward Y. Chang,et al.  Manifold learning, a promised land or work in progress? , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[32]  Christopher M. Bishop,et al.  A Hierarchical Latent Variable Model for Data Visualization , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  T. Hastie,et al.  Principal Curves , 2007 .

[34]  Peter Tiño,et al.  Hierarchical GTM: Constructing Localized Nonlinear Projection Manifolds in a Principled Way , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  N. Kambhatla.  Local models and Gaussian mixture models for statistical data processing , 1996 .

[36]  J. Michael Herrmann,et al.  Retinotopy and spatial phase in topographic maps , 2000, Neurocomputing.

[37]  Klaus Obermayer,et al.  Self-organizing maps: Generalizations and new optimization techniques , 1998, Neurocomputing.

[38]  Vladimir Cherkassky,et al.  Self-Organization as an Iterative Kernel Smoothing Process , 1995, Neural Computation.

[39]  M. Kramer.  Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[40]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[41]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[42]  Samuel Kaski,et al.  Self-Organized Formation of Various Invariant-Feature Filters in the Adaptive-Subspace SOM , 1997, Neural Computation.

[43]  IItevor Hattie.  Principal Curves and Surfaces , 1984 .