Batch and median neural gas

Neural Gas (NG) constitutes a very robust clustering algorithm given Euclidean data which does not suffer from the problem of local minima like simple vector quantization, or topological restrictions like the self-organizing map. Based on the cost function of NG, we introduce a batch variant of NG which shows much faster convergence and which can be interpreted as an optimization of the cost function by the Newton method. This formulation has the additional benefit that, based on the notion of the generalized median in analogy to Median SOM, a variant for non-vectorial proximity data can be introduced. We prove convergence of batch and median versions of NG, SOM, and k-means in a unified formulation, and we investigate the behavior of the algorithms in several experiments.

[1]  M. Vingron,et al.  Quantifying the local reliability of a sequence alignment. , 1996, Protein engineering.

[2]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[3]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[4]  Samuel Kaski,et al.  Principle of Learning Metrics for Exploratory Data Analysis , 2004, J. VLSI Signal Process..

[5]  Marie Cottrell,et al.  Advantages and drawbacks of the Batch Kohonen algorithm , 2002, ESANN.

[6]  Klaus Obermayer,et al.  Self-organizing maps and clustering methods for matrix data , 2004, Neural Networks.

[7]  Joydeep Ghosh,et al.  A Unified Framework for Model-based Clustering , 2003, J. Mach. Learn. Res..

[8]  Horst Bunke,et al.  Applications of approximate string matching to 2D shape recognition , 1993, Pattern Recognit..

[9]  Thomas Martinetz,et al.  Topology representing networks , 1994, Neural Networks.

[10]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[11]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[12]  Marie Cottrell,et al.  SOM-based algorithms for qualitative variables , 2004, Neural Networks.

[13]  Samuel Kaski,et al.  Principle of Learning Metrics for Data Analysis , 2004 .

[14]  Yoshua Bengio,et al.  Convergence Properties of the K-Means Algorithms , 1994, NIPS.

[15]  Frank-Michael Schleif,et al.  Fuzzy Labeled Neural GAS for Fuzzy Classification , 2005 .

[16]  Horst Bunke,et al.  Self-organizing map for clustering in the graph domain , 2002, Pattern Recognit. Lett..

[17]  E. Granum,et al.  Quantitative analysis of 6985 digitized trypsin G ‐banded human metaphase chromosomes , 1980, Clinical genetics.

[18]  Klaus Obermayer,et al.  A Stochastic Self-Organizing Map for Proximity Data , 1999, Neural Computation.

[19]  Thomas Villmann,et al.  Supervised Neural Gas with General Similarity Measure , 2005, Neural Processing Letters.

[20]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[21]  Panu Somervuo,et al.  How to make large self-organizing maps for nonvectorial data , 2002, Neural Networks.

[22]  Panu Somervuo.  Online algorithm for the self-organizing map of symbol strings , 2004, Neural Networks.

[23]  Tom Heskes,et al.  Self-organizing maps, vector quantization, and mixture modeling , 2001, IEEE Trans. Neural Networks.

[24]  Gilles Pagès,et al.  Theoretical aspects of the SOM algorithm , 1998, Neurocomputing.

[25]  Yizong Cheng.  Convergence and Ordering of Kohonen's Batch Map , 1997, Neural Computation.

[26]  Fabrice Rossi,et al.  Fast Algorithm and Implementation of Dissimilarity Self-Organizing Maps , 2006, Neural Networks.

[27]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[28]  Thomas Villmann,et al.  Topology preservation in self-organizing feature maps: exact definition and measurement , 1997, IEEE Trans. Neural Networks.

[29]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[30]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[31]  Fabrice Rossi,et al.  A Fast Algorithm for the Self-Organizing Map on Dissimilarity Data , 2005 .

[32]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[33]  David G. Stork,et al.  Pattern Classification , 1973 .

[34]  Gabriela Andreu,et al.  Selecting the toroidal self-organizing feature maps (TSOFM) best organized to object recognition , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).