DropConnect for Evaluation of Classification Stability in Learning Vector Quantization

In this paper we consider DropOut/DropConnect techniques known from deep neural networks to evaluate the stability of learning vector quantization classifiers (LVQ). For this purpose, we consider the LVQ as a multilayer network and transfer the respective concepts to LVQ. Particularly, we consider the output as a stochastic ensemble such that an information theoretic measure is obtained to judge the stability level.

[1]  David A. Landgrebe,et al.  Signal Theory Methods in Multispectral Remote Sensing , 2003 .

[2]  Thomas Villmann,et al.  Types of (dis-)similarities and adaptive mixtures thereof for improved classification learning , 2017, Neurocomputing.

[3]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[4]  Michel Verleysen,et al.  Representation of functional data in neural networks , 2005, Neurocomputing.

[5]  Thomas Villmann,et al.  Prototype-based models in machine learning. , 2016, Wiley interdisciplinary reviews. Cognitive science.

[6]  Thomas Villmann.  Learning Vector Quantization Methods for Interpretable Classification Learning and Multilayer Networks , 2018, IJCCI.

[7]  Thomas Villmann,et al.  Activation Functions for Generalized Learning Vector Quantization - A Performance Comparison , 2019, ArXiv.

[8]  Michael Biehl,et al.  Adaptive Relevance Matrices in Learning Vector Quantization , 2009, Neural Computation.

[9]  K. Ming Leung,et al.  Learning Vector Quantization , 2017, Encyclopedia of Machine Learning and Data Mining.

[10]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[11]  Thomas Villmann,et al.  Kernelized vector quantization in gradient-descent learning , 2015, Neurocomputing.

[12]  Klaus Obermayer,et al.  Soft Learning Vector Quantization , 2003, Neural Computation.

[13]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[14]  Michael Biehl,et al.  Empirical evaluation of gradient methods for matrix learning vector quantization , 2017, 2017 12th International Workshop on Self-Organizing Maps and Learning Vector Quantization, Clustering and Data Visualization (WSOM).

[15]  Thomas Villmann,et al.  Self-Adjusting Reject Options in Prototype Based Classification , 2016, WSOM.

[16]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.

[17]  Thomas Villmann,et al.  Can Learning Vector Quantization be an Alternative to SVM and Deep Learning? - Recent Trends and Advanced Variants of Learning Vector Quantization for Classification Learning , 2017, J. Artif. Intell. Soft Comput. Res..

[18]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[19]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..