Median variants of learning vector quantization for learning of dissimilarity data

Abstract Exemplar based techniques such as affinity propagation represent data in terms of typical exemplars. This has two benefits: (i) the resulting models are directly interpretable by humans since representative exemplars can be inspected in the same way as data points, (ii) the model can be applied to any dissimilarity measure including non-Euclidean or non-metric settings. Most exemplar based techniques have been proposed in the unsupervised setting only, such that their performance in supervised learning tasks can be weak depending on the given data. We address the problem of learning exemplar-based models for general dissimilarity data in a discriminative framework in this contribution. For this purpose, we consider variants of Kohonen׳s learning vector quantization model to handle data with only dissimilarities between available. Here the exemplars are the prototypes. The resulting classifiers represent data in terms of sparse models thereby reaching state-of-the art results in benchmarks. For a real world data set in the field of veterinary medicine we report promising results.

[1]  Barbara Hammer,et al.  Relevance learning in generative topographic mapping , 2011, Neurocomputing.

[2]  Trinad Chakraborty,et al.  Rapid Identification and Typing of Listeria Species by Matrix-Assisted Laser Desorption Ionization-Time of Flight Mass Spectrometry , 2008, Applied and Environmental Microbiology.

[3]  Liwei Wang,et al.  Theory and Algorithm for Learning with Dissimilarity Functions , 2009, Neural Computation.

[4]  Thomas Villmann,et al.  Relevance LVQ versus SVM , 2004, ICAISC.

[5]  Michael E. Tipping Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[6]  Barbara Hammer,et al.  Graph-Based Representation of Symbolic Musical Data , 2009, GbRPR.

[7]  Paulo J. G. Lisboa,et al.  Towards interpretable classifiers with blind signal separation , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[8]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[9]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.

[10]  Thomas Villmann,et al.  Kernelized vector quantization in gradient-descent learning , 2015, Neurocomputing.

[11]  R. Schatten,et al.  A theory of cross-spaces , 1950 .

[12]  Klaus Obermayer,et al.  Soft Learning Vector Quantization , 2003, Neural Computation.

[13]  Thomas Villmann,et al.  Efficient Kernelized Prototype Based Classification , 2011, Int. J. Neural Syst..

[14]  Jennifer Sturgis,et al.  Portable bacterial identification system based on elastic light scatter patterns , 2012, Journal of biological engineering.

[15]  Euiwon Bae,et al.  Label‐free identification of bacterial microcolonies via elastic scattering , 2011, Biotechnology and bioengineering.

[16]  James J. Thomas,et al.  Defining Insight for Visual Analytics , 2009, IEEE Computer Graphics and Applications.

[17]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[18]  Paulo J. G. Lisboa,et al.  Research directions in interpretable machine learning models , 2013, ESANN.

[19]  Frank-Michael Schleif,et al.  Learning vector quantization for (dis-)similarities , 2014, Neurocomputing.

[20]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[21]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[22]  T. Maier,et al.  Fast and reliable MALDI-TOF MS–based microorganism identification , 2006 .

[23]  Thomas Villmann,et al.  Aspects in Classification Learning - Review of Recent Developments in Learning Vector Quantization , 2014 .

[24]  Barbara Hammer,et al.  Relational Generative Topographic Map , 2010, ESANN.

[25]  Friedhelm Schwenker,et al.  Artificial Neural Networks in Pattern Recognition: 5th INNS IAPR TC 3 GIRPR Workshop, ANNPR 2012, Trento, Italy, September 17-19, 2012 , 2012 .

[26]  Thomas Villmann,et al.  Supervised median clustering , 2006 .

[27]  Michael Biehl,et al.  Distance Learning in Discriminative Vector Quantization , 2009, Neural Computation.

[28]  Thomas Villmann,et al.  Generalized relevance learning vector quantization , 2002, Neural Networks.

[29]  Alessandro Sperduti,et al.  Mining Structured Data , 2010, IEEE Computational Intelligence Magazine.

[30]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[31]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[32]  Deniz Erdogmus,et al.  Information Theoretic Learning , 2005, Encyclopedia of Artificial Intelligence.

[33]  Robert P. W. Duin,et al.  The Dissimilarity Representation for Pattern Recognition - Foundations and Applications , 2005, Series in Machine Perception and Artificial Intelligence.

[34]  Alessio Micheli,et al.  A general framework for unsupervised processing of structured data , 2004, Neurocomputing.

[35]  Thomas Villmann,et al.  Derivatives of Pearson Correlation for Gradient-based Analysis of Biomedical Data , 2008, Inteligencia Artif..

[36]  Horst Bunke,et al.  Edit distance-based kernel functions for structural pattern classification , 2006, Pattern Recognit..

[37]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[38]  James C. Bezdek,et al.  Relational duals of the c-means clustering algorithms , 1989, Pattern Recognit..

[39]  Thomas Gärtner,et al.  Guest editors’ introduction: special issue on mining and learning with graphs , 2009, Machine Learning.

[40]  Maya R. Gupta,et al.  Similarity-based Classification: Concepts and Algorithms , 2009, J. Mach. Learn. Res..

[41]  Thomas Villmann,et al.  Non-euclidean Principal Component Analysis for Matrices by Hebbian Learning , 2014, ICAISC.

[42]  Thomas Villmann,et al.  Supervised Neural Gas with General Similarity Measure , 2005, Neural Processing Letters.

[43]  Thomas Villmann,et al.  Generative versus Discriminative Prototype Based Classification , 2014, WSOM.

[44]  Thomas Villmann,et al.  On the Generalization Ability of GRLVQ Networks , 2005, Neural Processing Letters.

[45]  Teuvo Kohonen,et al.  Learning vector quantization , 1998 .

[46]  Ponnuthurai Nagaratnam Suganthan,et al.  A novel kernel prototype-based learning algorithm , 2004, ICPR 2004.

[47]  Thomas Villmann,et al.  Divergence-Based Vector Quantization , 2011, Neural Computation.

[48]  E. Granum,et al.  Quantitative analysis of 6985 digitized trypsin G ‐banded human metaphase chromosomes , 1980, Clinical genetics.

[49]  Bernard Haasdonk,et al.  Feature space interpretation of SVMs with indefinite kernels , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  J. Stark,et al.  Network motifs: structure does not determine function , 2006, BMC Genomics.

[51]  Heiko Wersing,et al.  Online figure-ground segmentation with adaptive metrics in generalized LVQ , 2009, Neurocomputing.

[52]  J. Keller,et al.  Clustering in ordered dissimilarity data , 2009 .

[53]  Klaus Obermayer,et al.  Soft nearest prototype classification , 2003, IEEE Trans. Neural Networks.

[54]  James C. Bezdek,et al.  Nerf c-means: Non-Euclidean relational fuzzy clustering , 1994, Pattern Recognit..

[55]  Thomas Villmann,et al.  Non-Euclidean principal component analysis by Hebbian learning , 2015, Neurocomputing.

[56]  Martin A. Riedmiller,et al.  Incremental GRLVQ: Learning relevant features for 3D object recognition , 2008, Neurocomputing.

[57]  Michael Biehl,et al.  Urine Steroid Metabolomics as a Biomarker Tool for Detecting Malignancy in Adrenal Tumors , 2011, The Journal of clinical endocrinology and metabolism.

[58]  Jean-Louis Lassez,et al.  Similarity Based Classification , 2003, IDA.

[59]  P. Grassberger,et al.  Sequence Alignment, Mutual Information, and Dissimilarity Measures for Constructing Phylogenies , 2010, PloS one.

[60]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[61]  James C. Bezdek,et al.  Numerical comparison of the RFCM and AP algorithms for clustering relational data , 1991 .

[62]  J. Paul Robinson,et al.  Optical forward-scattering for detection of Listeria monocytogenes and other Listeria species. , 2007, Biosensors & bioelectronics.

[63]  Nathan J Hillson,et al.  DeviceEditor visual biological CAD canvas , 2012, Journal of Biological Engineering.

[64]  Barbara Hammer,et al.  Topographic Mapping of Large Dissimilarity Data Sets , 2010, Neural Computation.

[65]  Michael Biehl,et al.  Adaptive Relevance Matrices in Learning Vector Quantization , 2009, Neural Computation.

[66]  James M. Keller,et al.  Improvements to the relational fuzzy c-means clustering algorithm , 2014, Pattern Recognit..

[67]  Thomas Villmann,et al.  Learning vector quantization for heterogeneous structured data , 2010, ESANN.

[68]  Paul M. B. Vitányi,et al.  Clustering by compression , 2003, IEEE Transactions on Information Theory.

[69]  Klaus Obermayer,et al.  Structure Spaces , 2009, J. Mach. Learn. Res..

[70]  Michael Biehl,et al.  Dynamics and Generalization Ability of LVQ Algorithms , 2007, J. Mach. Learn. Res..

[71]  Barbara Hammer,et al.  Kernel Robust Soft Learning Vector Quantization , 2012, ANNPR.