Supervised Neural Gas with General Similarity Measure

Prototype based classification offers intuitive and sparse models with excellent generalization ability. However, these models usually crucially depend on the underlying Euclidian metric; moreover, online variants likely suffer from the problem of local optima. We here propose a generalization of learning vector quantization with three additional features: (I) it directly integrates neighborhood cooperation, hence is less affected by local optima; (II) the method can be combined with any differentiable similarity measure whereby metric parameters such as relevance factors of the input dimensions can automatically be adapted according to the given data; (III) it obeys a gradient dynamics hence shows very robust behavior, and the chosen objective is related to margin optimization.

[1]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[2]  Thomas Martinetz,et al.  Topology representing networks , 1994, Neural Networks.

[3]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[4]  Gunnar Rätsch,et al.  New Methods for Splice Site Recognition , 2002, ICANN.

[5]  David Haussler,et al.  A Discriminative Framework for Detecting Remote Protein Homologies , 2000, J. Comput. Biol..

[6]  C. Gielen,et al.  Neural computation and self-organizing maps, an introduction , 1993 .

[7]  Biing-Hwang Juang,et al.  Discriminative learning for minimum error classification [pattern recognition] , 1992, IEEE Trans. Signal Process..

[8]  Thomas Villmann,et al.  Generalized relevance learning vector quantization , 2002, Neural Networks.

[9]  Gérard Dreyfus,et al.  Training wavelet networks for nonlinear dynamic input-output modeling , 1998, Neurocomputing.

[10]  Marc Toussaint,et al.  Extracting Motion Primitives from Natural Handwriting Data , 2006, ICANN.

[11]  W. Peizhuang.  Pattern Recognition with Fuzzy Objective Function Algorithms (James C. Bezdek) , 1983 .

[12]  Gert Pfurtscheller,et al.  Automated feature selection with a distinction sensitive learning vector quantizer , 1996, Neurocomputing.

[13]  Nikos A. Vlassis,et al.  A Greedy EM Algorithm for Gaussian Mixture Learning , 2002, Neural Processing Letters.

[14]  Barbara Hammer,et al.  A Note on the Universal Approximation Capability of Support Vector Machines , 2003, Neural Processing Letters.

[15]  Klaus Obermayer,et al.  Self-organizing maps: Generalizations and new optimization techniques , 1998, Neurocomputing.

[16]  Klaus Obermayer,et al.  Soft Learning Vector Quantization , 2003, Neural Computation.

[17]  Klaus Schulten,et al.  Self-organizing maps: ordering, convergence properties and energy functions , 1992, Biological Cybernetics.

[18]  Thomas Villmann,et al.  Neural maps and topographic vector quantization , 1999, Neural Networks.

[19]  Wlodzislaw Duch,et al.  A new methodology of extraction, optimization and application of crisp and fuzzy logical rules , 2001, IEEE Trans. Neural Networks.

[20]  Giuseppe Patanè,et al.  The enhanced LBG algorithm , 2001, Neural Networks.

[21]  Teuvo Kohonen,et al.  Learning vector quantization , 1998 .

[22]  Thomas Villmann,et al.  Growing a hypercubical output space in a self-organizing feature map , 1997, IEEE Trans. Neural Networks.

[23]  Tom Heskes,et al.  Self-organizing maps, vector quantization, and mixture modeling , 2001, IEEE Trans. Neural Networks.

[24]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[25]  Samuel Kaski,et al.  A Topography-Preserving Latent Variable Model with Learning Metrics , 2001, WSOM.

[26]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[27]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.

[28]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[29]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[30]  Koby Crammer,et al.  Margin Analysis of the LVQ Algorithm , 2002, NIPS.

[31]  Samuel Kaski,et al.  Bankruptcy analysis with self-organizing maps in learning metrics , 2001, IEEE Trans. Neural Networks.

[32]  T. Heskes.  Energy functions for self-organizing maps , 1999 .

[33]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[34]  M. R. Spiegel.  E and M , 1981 .

[35]  Barbara Hammer,et al.  Relevance determination in Learning Vector Quantization , 2001, ESANN.

[36]  Thomas Villmann,et al.  Learning Vector Quantization for Multimodal Data , 2002, ICANN.

[37]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[38]  Thomas Villmann,et al.  Topology preservation in self-organizing feature maps: exact definition and measurement , 1997, IEEE Trans. Neural Networks.

[39]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Thomas Villmann,et al.  Neural maps in remote sensing image analysis , 2003, Neural Networks.

[41]  R. Davé.  FUZZY SHELL-CLUSTERING AND APPLICATIONS TO CIRCLE DETECTION IN DIGITAL IMAGES , 1990 .

[42]  A. Sato,et al.  An Analysis of Convergence in Generalized LVQ , 1998 .

[43]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[44]  Barbara Hammer,et al.  Generalized Relevance LVQ for Time Series , 2001, ICANN.