A General Framework for Dimensionality-Reducing Data Visualization Mapping

In recent years, a wealth of dimension-reduction techniques for data visualization and preprocessing has been established. Nonparametric methods require additional effort for out-of-sample extensions, because they provide only a mapping of a given finite set of points. In this letter, we propose a general view on nonparametric dimension reduction based on the concept of cost functions and properties of the data. Based on this general principle, we transfer nonparametric dimension reduction to explicit mappings of the data manifold such that direct out-of-sample extensions become possible. Furthermore, this concept offers the possibility of investigating the generalization ability of data visualization to new data points. We demonstrate the approach based on a simple global linear mapping, as well as prototype-based local linear mappings. In addition, we can bias the functional form according to given auxiliary information. This leads to explicit supervised visualization mappings with discriminative properties comparable to state-of-the-art approaches.

[1]  Jorma Laaksonen,et al.  LVQ_PAK: The Learning Vector Quantization Program Package , 1996 .

[2]  Samuel Kaski,et al.  Bankruptcy analysis with self-organizing maps in learning metrics , 2001, IEEE Trans. Neural Networks.

[3]  Le Song,et al.  Colored Maximum Variance Unfolding , 2007, NIPS.

[4]  Barbara Hammer,et al.  Visualizing the quality of dimensionality reduction , 2013, ESANN.

[5]  Heiko Hoffmann,et al.  An extension of neural gas to local PCA , 2004, Neurocomputing.

[6]  Michel Verleysen,et al.  Nonlinear Dimensionality Reduction , 2021, Computer Vision.

[7]  Daniel A. Keim,et al.  Visual Analytics: Scope and Challenges , 2008, Visual Data Mining.

[8]  Axel Wismüller,et al.  Adaptive local dissimilarity measures for discriminative dimension reduction of labeled data , 2010, Neurocomputing.

[9]  Peter L. Bartlett,et al.  Rademacher and Gaussian Complexities: Risk Bounds and Structural Results , 2003, J. Mach. Learn. Res..

[10]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[11]  Michael Biehl,et al.  Adaptive Relevance Matrices in Learning Vector Quantization , 2009, Neural Computation.

[12]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[13]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[14]  Thomas Villmann,et al.  Batch and median neural gas , 2006, Neural Networks.

[15]  Yee Whye Teh,et al.  Automatic Alignment of Local Representations , 2002, NIPS.

[16]  Geoffrey E. Hinton,et al.  Stochastic Neighbor Embedding , 2002, NIPS.

[17]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[18]  W. Torgerson.  Multidimensional scaling: I. Theory and method , 1952 .

[19]  Barbara Hammer,et al.  Linear basis-function t-SNE for fast nonlinear dimensionality reduction , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[20]  Yeuvo Jphonen,et al.  Self-Organizing Maps , 1995 .

[21]  Thomas L. Griffiths,et al.  Parametric Embedding for Class Visualization , 2004, Neural Computation.

[22]  Laurens van der Maaten,et al.  Learning a Parametric Embedding by Preserving Local Structure , 2009, AISTATS.

[23]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[24]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[25]  Michael H. Böhlen,et al.  Visual Data Mining - Theory, Techniques and Tools for Visual Analytics , 2008, Visual Data Mining.

[26]  Hau-San Wong,et al.  Kernel clustering-based discriminant analysis , 2007, Pattern Recognit..

[27]  Kilian Q. Weinberger,et al.  An Introduction to Nonlinear Dimensionality Reduction by Maximum Variance Unfolding , 2006, AAAI.

[28]  Michel Verleysen,et al.  Quality assessment of dimensionality reduction: Rank-based criteria , 2009, Neurocomputing.

[29]  Barbara Hammer,et al.  On the effect of clustering on quality assessment measures for dimensionality reduction , 2010, NIPS 2010.

[30]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[31]  Jarkko Venna,et al.  Information Retrieval Perspective to Nonlinear Dimensionality Reduction for Data Visualization , 2010, J. Mach. Learn. Res..

[32]  Geoffrey E. Hinton,et al.  Multiple Relational Embedding , 2004, NIPS.

[33]  Geoffrey C. Fox,et al.  Dimension reduction and visualization of large high-dimensional data via interpolation , 2010, HPDC '10.

[34]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[35]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[36]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[37]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[38]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[39]  Matthew Brand,et al.  Charting a Manifold , 2002, NIPS.

[40]  Michel Verleysen,et al.  Rank-based quality assessment of nonlinear dimensionality reduction , 2008, ESANN.

[41]  Barbara Hammer.  Challenges in Neural Computation , 2012, KI - Künstliche Intelligenz.

[42]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[43]  Miguel Á. Carreira-Perpiñán,et al.  The Elastic Embedding Algorithm for Dimensionality Reduction , 2010, ICML.

[44]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[45]  Samuel Kaski,et al.  Improved learning of Riemannian metrics for exploratory analysis [Neural Networks 17 (8–9) 1087–1100] , 2005 .

[46]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[47]  Zhi-Hua Zhou,et al.  Supervised nonlinear dimensionality reduction for visualization and classification , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[48]  Johan A. K. Suykens,et al.  Data Visualization and Dimensionality Reduction Using Kernel Maps With a Reference Point , 2008, IEEE Transactions on Neural Networks.