Limited Rank Matrix Learning, discriminative dimension reduction and visualization

We present an extension of the recently introduced Generalized Matrix Learning Vector Quantization algorithm. In the original scheme, adaptive square matrices of relevance factors parameterize a discriminative distance measure. We extend the scheme to matrices of limited rank corresponding to low-dimensional representations of the data. This allows to incorporate prior knowledge of the intrinsic dimension and to reduce the number of adaptive parameters efficiently. In particular, for very large dimensional data, the limitation of the rank can reduce computation time and memory requirements significantly. Furthermore, two- or three-dimensional representations constitute an efficient visualization method for labeled data sets. The identification of a suitable projection is not treated as a pre-processing step but as an integral part of the supervised training. Several real world data sets serve as an illustration and demonstrate the usefulness of the suggested method.

[1]  Thomas Villmann,et al.  Generalized relevance learning vector quantization , 2002, Neural Networks.

[2]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[3]  Marcel F. Jonkman,et al.  Adaptive Metrics for Content Based Image Retrieval in Dermatology , 2009, ESANN.

[4]  Michael Biehl,et al.  Dynamics and Generalization Ability of LVQ Algorithms , 2007, J. Mach. Learn. Res..

[5]  Vin de Silva,et al.  Reduction A Global Geometric Framework for Nonlinear Dimensionality , 2011 .

[6]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[7]  Matthew Brand,et al.  Charting a Manifold , 2002, NIPS.

[8]  M. Ringnér,et al.  Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks , 2001, Nature Medicine.

[9]  G. Celeux,et al.  Regularized Gaussian Discriminant Analysis through Eigenvalue Decomposition , 1996 .

[10]  Thomas Villmann,et al.  On the Generalization Ability of GRLVQ Networks , 2005, Neural Processing Letters.

[11]  Michael Biehl,et al.  Nonlinear Discriminative Data Visualization , 2009, ESANN.

[12]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[13]  Thomas Villmann,et al.  Generalized Derivative Based Kernelized Learning Vector Quantization , 2010, IDEAL.

[14]  Aapo Hyvärinen,et al.  17th European Symposium on Artificial Neural Networks (ESANN) , 2009 .

[15]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[16]  Maia Angelova,et al.  Gene expression Targeted projection pursuit for visualizing gene expression data classifications , 2006 .

[17]  Aidong Zhang,et al.  VizStruct: exploratory visualization for gene expression profiling , 2004, Bioinform..

[18]  Samuel Kaski,et al.  Bankruptcy analysis with self-organizing maps in learning metrics , 2001, IEEE Trans. Neural Networks.

[19]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[20]  Michael Biehl,et al.  Adaptive Relevance Matrices in Learning Vector Quantization , 2009, Neural Computation.

[21]  Jarkko Venna,et al.  Information Retrieval Perspective to Nonlinear Dimensionality Reduction for Data Visualization , 2010, J. Mach. Learn. Res..

[22]  Naphtali Rishe,et al.  Content-based image retrieval , 1995, Multimedia Tools and Applications.

[23]  Thomas Villmann,et al.  Supervised Neural Gas with General Similarity Measure , 2005, Neural Processing Letters.

[24]  Thomas L. Griffiths,et al.  Parametric Embedding for Class Visualization , 2004, Neural Computation.

[25]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[26]  Thomas Villmann,et al.  Neural maps in remote sensing image analysis , 2003, Neural Networks.

[27]  Xin Geng,et al.  Supervised nonlinear dimensionality reduction for visualization and classification , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[29]  Marcel F. Jonkman,et al.  Learning effective color features for content based image retrieval in dermatology , 2011, Pattern Recognit..

[30]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[31]  Le Song,et al.  Colored Maximum Variance Unfolding , 2007, NIPS.

[32]  Barbara Hammer,et al.  Relevance determination in Learning Vector Quantization , 2001, ESANN.

[33]  Axel Wismüller,et al.  Adaptive local dissimilarity measures for discriminative dimension reduction of labeled data , 2010, Neurocomputing.

[34]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[35]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[36]  Hau-San Wong,et al.  Kernel clustering-based discriminant analysis , 2007, Pattern Recognit..

[37]  Geoffrey E. Hinton,et al.  Multiple Relational Embedding , 2004, NIPS.

[38]  Dianne Cook,et al.  Projection Pursuit for Exploratory Supervised Classification , 2005 .

[39]  Masashi Sugiyama,et al.  Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis , 2007, J. Mach. Learn. Res..

[40]  David G. Stork,et al.  Pattern Classification , 1973 .

[41]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[42]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[43]  D. Botstein,et al.  A gene expression database for the molecular pharmacology of cancer , 2000, Nature Genetics.

[44]  J. Michael Cherry,et al.  Visualization of expression clusters using Sammon's non-linear mapping , 2001, Bioinform..

[45]  D. Botstein,et al.  Cluster analysis and display of genome-wide expression patterns. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[46]  Thomas Villmann,et al.  Regularization in Matrix Relevance Learning , 2010, IEEE Transactions on Neural Networks.

[47]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[48]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[49]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.