On the Generalization Ability of Prototype-Based Classifiers with Local Relevance Determination

We extend a recent variant of the prototype-based classifier learning vector quantization to a scheme which locally adapts relevance terms during learning. We derive explicit dimensionality-independent large-margin generalization bounds for this classifier and show that the method can be seen as margin maximizer.

[1]  Barbara Hammer,et al.  A Note on the Universal Approximation Capability of Support Vector Machines , 2003, Neural Processing Letters.

[2]  Michael Biehl,et al.  Learning vector quantization: The dynamics of winner-takes-all algorithms , 2006, Neurocomputing.

[3]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.

[4]  Thomas Martinetz,et al.  Topology representing networks , 1994, Neural Networks.

[5]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[6]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[7]  Barbara Hammer,et al.  Generalized Relevance LVQ for Time Series , 2001, ICANN.

[8]  Horst Bunke,et al.  Self-organizing map for clustering in the graph domain , 2002, Pattern Recognit. Lett..

[9]  Thomas Villmann,et al.  Neural maps in remote sensing image analysis , 2003, Neural Networks.

[10]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[11]  Vladimir Cherkassky,et al.  Comparison of adaptive methods for function estimation from samples , 1996, IEEE Trans. Neural Networks.

[12]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[13]  Johan A. K. Suykens,et al.  Automatic relevance determination for Least Squares Support Vector Machines classifiers , 2001, ESANN.

[14]  Samuel Kaski,et al.  A Topography-Preserving Latent Variable Model with Learning Metrics , 2001, WSOM.

[15]  R. Davé.  FUZZY SHELL-CLUSTERING AND APPLICATIONS TO CIRCLE DETECTION IN DIGITAL IMAGES , 1990 .

[16]  Olivier Bousquet,et al.  On the Complexity of Learning the Kernel Matrix , 2002, NIPS.

[17]  Thomas Villmann,et al.  Supervised Neural Gas with General Similarity Measure , 2005, Neural Processing Letters.

[18]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[19]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[20]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[21]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[22]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[23]  Panu Somervuo,et al.  How to make large self-organizing maps for nonvectorial data , 2002, Neural Networks.

[24]  Yves Grandvalet.  Anisotropic noise injection for input variables relevance determination , 2000, IEEE Trans. Neural Networks Learn. Syst..

[25]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Peter L. Bartlett,et al.  Rademacher and Gaussian Complexities: Risk Bounds and Structural Results , 2003, J. Mach. Learn. Res..

[27]  David Haussler,et al.  A Discriminative Framework for Detecting Remote Protein Homologies , 2000, J. Comput. Biol..

[28]  Klaus Obermayer,et al.  Soft Learning Vector Quantization , 2003, Neural Computation.

[29]  Koby Crammer,et al.  Margin Analysis of the LVQ Algorithm , 2002, NIPS.

[30]  Samuel Kaski,et al.  Bankruptcy analysis with self-organizing maps in learning metrics , 2001, IEEE Trans. Neural Networks.

[31]  T. Heskes.  Energy functions for self-organizing maps , 1999 .

[32]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[33]  A. Sato,et al.  An Analysis of Convergence in Generalized LVQ , 1998 .

[34]  Barbara Hammer,et al.  Metric adaptation and relevance learning in learning vector quantization , 2003 .

[35]  Thomas Villmann,et al.  Generalized relevance learning vector quantization , 2002, Neural Networks.

[36]  Vladimir Cherkassky,et al.  Statistical analysis of self-organization , 1995, Neural Networks.