Neighbor embedding XOM for dimension reduction and visualization

We present an extension of the Exploratory Observation Machine (XOM) for structure-preserving dimensionality reduction. Based on minimizing the Kullback-Leibler divergence of neighborhood functions in data and image spaces, this Neighbor Embedding XOM (NE-XOM) creates a link between fast sequential online learning known from topology-preserving mappings and principled direct divergence optimization approaches. We quantitatively evaluate our method on real-world data using multiple embedding quality measures. In this comparison, NE-XOM performs as a competitive trade-off between high embedding quality and low computational expense, which motivates its further use in real-world settings throughout science and engineering.

[1]  Per-Olof Persson,et al.  A Simple Mesh Generator in MATLAB , 2004, SIAM Rev..

[2]  Matthew Brand,et al.  Charting a Manifold , 2002, NIPS.

[3]  M. Vingron,et al.  Quantifying the local reliability of a sequence alignment. , 1996, Protein engineering.

[4]  Michel Verleysen,et al.  Rank-based quality assessment of nonlinear dimensionality reduction , 2008, ESANN.

[5]  Thomas Villmann,et al.  Discriminative visualization by limited rank matrix learning , 2008 .

[6]  Michel Verleysen,et al.  Recent Advances in Nonlinear Dimensionality Reduction, Manifold and Topological Learning , 2010, ESANN.

[7]  Colin Fyfe,et al.  Bregman Divergences and the Self Organising Map , 2008, IDEAL.

[8]  Michel Verleysen,et al.  Locally Linear Embedding versus Isotop , 2003, ESANN.

[9]  Nico Karssemeijer,et al.  Medical Imaging 2009: Computer-aided Diagnosis , 2009 .

[10]  M. V. Velzen,et al.  Self-organizing maps , 2007 .

[11]  Axel Wismüller.  A Computational Framework for Nonlinear Dimensionality Reduction and Clustering , 2009, WSOM.

[12]  Axel Wismüller.  Exploration-Organized Morphogenesis (XOM): A general framework for learning by self-organization , 2001 .

[13]  Thomas Villmann,et al.  Divergence based Learning Vector Quantization , 2010, ESANN.

[14]  Thomas Villmann,et al.  Fuzzy classification by fuzzy labeled neural gas , 2006, Neural Networks.

[15]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[16]  Colin Fyfe,et al.  Bregman Divergences and Multi-dimensional Scaling , 2009, ICONIP.

[17]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[18]  Mihoko Minami,et al.  Robust Blind Source Separation by Beta Divergence , 2002, Neural Computation.

[19]  Michel Verleysen,et al.  Nonlinear Dimensionality Reduction , 2021, Computer Vision.

[20]  Klaus Obermayer,et al.  Classi cation on Pairwise Proximity , 2007 .

[21]  Jarkko Venna,et al.  Dimensionality reduction for visual exploration of similarity structures , 2007 .

[22]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[23]  Claus Bahlmann,et al.  Learning with Distance Substitution Kernels , 2004, DAGM-Symposium.

[24]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[25]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[26]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[27]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[28]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[29]  Deniz Erdoğmuş,et al.  Vector-quantization by density matching in the minimum Kullback-Leibler divergence sense , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[30]  Yee Whye Teh,et al.  Automatic Alignment of Local Representations , 2002, NIPS.

[31]  Geoffrey E. Hinton,et al.  Stochastic Neighbor Embedding , 2002, NIPS.

[32]  Axel Wismüller,et al.  Computational intelligence in biomedical imaging: multidimensional analysis of spatio-temporal patterns , 2011, Computer Science - Research and Development.

[33]  Thomas Villmann,et al.  Exploratory Observation Machine (XOM) with Kullback-Leibler Divergence for Dimensionality Reduction and Visualization , 2010, ESANN.

[34]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[35]  T. Heskes.  Energy functions for self-organizing maps , 1999 .

[36]  Axel Wismüller.  The exploration machine: a novel method for analyzing high-dimensional data in computer-aided diagnosis , 2009, Medical Imaging.

[37]  Axel Wismüller,et al.  Adaptive local dissimilarity measures for discriminative dimension reduction of labeled data , 2010, Neurocomputing.

[38]  Deniz Erdogmus,et al.  Vector quantization using information theoretic concepts , 2005, Natural Computing.

[39]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[40]  Michel Verleysen,et al.  Quality assessment of dimensionality reduction: Rank-based criteria , 2009, Neurocomputing.

[42]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..