Border-Sensitive Learning in Kernelized Learning Vector Quantization

Prototype based classification approaches are powerful classifiers for class discrimination of vectorial data. Famous examples are learning vector quantization models (LVQ) and support vector machines (SVMs). In this paper we propose the application of kernel distances in LVQ such that the LVQ-algorithm can handle the data in a topologically equivalent data space compared to the feature mapping space in SVMs. Further, we provide strategies to force the LVQ-prototypes to be class border sensitive. In this way an alternative to SVMs based on Hebbian learning is established. After presenting the theoretical background, we demonstrate the abilities of the model for an illustrative toy example and for the more challenging task of classification of Wilson's disease patients according to their neurophysiological impairments.

[1]  Lakhmi C. Jain,et al.  Innovations in Neural Information Paradigms and Applications , 2009, Innovations in Neural Information Paradigms and Applications.

[2]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[3]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[4]  Thomas Villmann,et al.  Functional relevance learning in generalized learning vector quantization , 2012, Neurocomputing.

[5]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[6]  T. Villmann,et al.  Elektrophysiologisches Schädigungsprofil von Patienten mit einem Morbus Wilson , 2003, Der Nervenarzt.

[7]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.

[8]  Thomas Villmann,et al.  Efficient Kernelized Prototype Based Classification , 2011, Int. J. Neural Syst..

[9]  Ralf Der,et al.  The "Perceptual Magnet" Effect: A Model Based on Self-Organizing Feature Maps , 1995 .

[10]  T. Villmann,et al.  Correlation between automated writing movements and striatal dopaminergic innervation in patients with Wilson's disease , 2002, Journal of Neurology.

[11]  M.Kleinberg Jon,et al.  Advances in Self-Organizing Maps, 7th International Workshop, WSOM 2009, St. Augustine, FL, USA, June 8-10, 2009. Proceedings , 2009, WSOM.

[12]  Michael Biehl,et al.  Adaptive Relevance Matrices in Learning Vector Quantization , 2009, Neural Computation.

[13]  Thomas Villmann,et al.  Derivatives of Pearson Correlation for Gradient-based Analysis of Biomedical Data , 2008, Inteligencia Artif..

[14]  Michael Biehl,et al.  Distance Learning in Discriminative Vector Quantization , 2009, Neural Computation.

[15]  T. Villmann,et al.  Klassifikation des Morbus Wilson auf der Basis neurophysiologischer Parameter , 2005, Der Nervenarzt.

[16]  Thomas Villmann,et al.  Gradient Based Learning in Vector Quantization Using Differentiable Kernels , 2012, WSOM.

[17]  Thomas Villmann,et al.  Metric Learning for Prototype-Based Classification , 2009, Innovations in Neural Information Paradigms and Applications.

[18]  T. Villmann,et al.  Comparison of clinical types of Wilson's disease and glucose metabolism in extrapyramidal motor brain regions , 2002, Journal of Neurology.

[19]  Thomas Villmann,et al.  Divergence-Based Vector Quantization , 2011, Neural Computation.

[20]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[21]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[22]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[23]  Michael Biehl,et al.  Window-Based Example Selection in Learning Vector Quantization , 2010, Neural Computation.

[24]  Thomas Villmann,et al.  Investigation of fine-motor disturbances in Wilson's disease , 2003, Neurological Sciences.

[25]  Shaomin Mu,et al.  Using Cooperative Clustering to Solve Multiclass Problems , 2011 .

[26]  Thomas Villmann,et al.  Limited Rank Matrix Learning, discriminative dimension reduction and visualization , 2012, Neural Networks.

[27]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[28]  Thomas Villmann,et al.  Generalized relevance learning vector quantization , 2002, Neural Networks.

[29]  Troels Andreasen,et al.  Foundations of Intelligent Systems , 2014, Lecture Notes in Computer Science.

[30]  T. Villmann,et al.  Classification of fine‐motoric disturbances in Wilson's disease using artificial neural networks , 2005, Acta neurologica Scandinavica.

[31]  Thomas Villmann,et al.  Regularization in Matrix Relevance Learning , 2010, IEEE Transactions on Neural Networks.

[32]  Thomas Villmann,et al.  Supervised Neural Gas with General Similarity Measure , 2005, Neural Processing Letters.