Information preserving and locally isometric&conformal embedding via Tangent Manifold Learning

In many Data Analysis tasks, one deals with data that are presented in high-dimensional spaces. In practice original high-dimensional data are transformed into lower-dimensional representations (features) preserving certain subject-driven data properties such as distances or geodesic distances, angles, etc. Preserving as much as possible available information contained in the original high-dimensional data is also an important and desirable property of the representation. The real-world high-dimensional data typically lie on or near a certain unknown low-dimensional manifold (Data manifold) embedded in an ambient high-dimensional `observation' space, so in this article we assume this Manifold assumption to be fulfilled. An exact isometric manifold embedding in a low-dimensional space is possible in certain special cases only, so we consider the problem of constructing a `locally isometric and conformal' embedding, which preserves distances and angles between close points. We propose a new geometrically motivated locally isometric and conformal representation method, which employs Tangent Manifold Learning technique consisting in sample-based estimation of tangent spaces to the unknown Data manifold. In numerical experiments, the proposed method compares favourably with popular Manifold Learning methods in terms of isometric and conformal embedding properties as well as of accuracy of Data manifold reconstruction from the sample.

[1]  Joshua B. Tenenbaum,et al.  Global Versus Local Methods in Nonlinear Dimensionality Reduction , 2002, NIPS.

[2]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[3]  Daniel N. Kaslovsky,et al.  Non-Asymptotic Analysis of Tangent Space Perturbation , 2011 .

[4]  Lior Wolf,et al.  Learning over Sets using Kernel Principal Angles , 2003, J. Mach. Learn. Res..

[5]  Kilian Q. Weinberger,et al.  Spectral Methods for Dimensionality Reduction , 2006, Semi-Supervised Learning.

[6]  Jorge S. Marques,et al.  Non-linear dimension reduction with tangent bundle approximation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[7]  Lawrence Cayton,et al.  Algorithms for manifold learning , 2005 .

[8]  Yun Fu,et al.  Conformal Embedding Analysis with Local Graph Modeling on the Unit Hypersphere , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Jianwei Yin,et al.  Incremental Manifold Learning Via Tangent Space Alignment , 2006, ANNPR.

[10]  Gang Kou,et al.  A kernel entropy manifold learning approach for financial data analysis , 2014, Decis. Support Syst..

[11]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[12]  D. Donoho,et al.  Hessian Eigenmaps : new locally linear embedding techniques for high-dimensional data , 2003 .

[13]  Larry A. Wasserman,et al.  Minimax Manifold Estimation , 2010, J. Mach. Learn. Res..

[14]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[15]  Shancang Li,et al.  A Novel Manifold Learning Algorithm for Localization Estimation in Wireless Sensor Networks , 2007, IEICE Trans. Commun..

[16]  Nicolas Le Roux,et al.  Learning Eigenfunctions Links Spectral Embedding and Kernel PCA , 2004, Neural Computation.

[17]  Ross T. Whitaker,et al.  On the Manifold Structure of the Space of Brain Images , 2009, MICCAI.

[18]  Ross T. Whitaker,et al.  Manifold modeling for brain population analysis , 2010, Medical Image Anal..

[19]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[20]  Alexander P. Kuleshov,et al.  Manifold Learning: Generalization Ability and Tangent Proximity , 2013, Int. J. Softw. Informatics.

[21]  Pascal Frossard,et al.  Tangent space estimation for smooth embeddings of Riemannian manifolds , 2012 .

[22]  Vin de Silva,et al.  Unsupervised Learning of Curved Manifolds , 2003 .

[23]  David L. Donoho,et al.  Image Manifolds which are Isometric to Euclidean Space , 2005, Journal of Mathematical Imaging and Vision.

[24]  Alexander P. Kuleshov,et al.  Manifold Learning: Generalization Ability and Tangent Proximity , 2013, Int. J. Softw. Informatics.

[25]  Alexander P. Kuleshov,et al.  Tangent Bundle Manifold Learning via Grassmann&Stiefel Eigenmaps , 2012, ArXiv.

[26]  Matthew Brand,et al.  Charting a Manifold , 2002, NIPS.

[27]  Xiaofei He,et al.  Tangent space learning and generalization , 2011 .

[28]  Rui Li,et al.  Simultaneous Learning of Nonlinear Manifold and Dynamical Models for High-dimensional Time Series , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[29]  Michel Verleysen,et al.  Quality assessment of nonlinear dimensionality reduction based on K-ary neighborhoods , 2008, FSDM.

[30]  Hyunsoo Kim,et al.  Distance Preserving Dimension Reduction for Manifold Learning , 2007, SDM.

[31]  Pascal Vincent,et al.  The Manifold Tangent Classifier , 2011, NIPS.

[32]  H. Sebastian Seung,et al.  The Manifold Ways of Perception , 2000, Science.

[33]  Yoshua Bengio,et al.  Non-Local Manifold Tangent Learning , 2004, NIPS.

[34]  Robert Pless,et al.  A Survey of Manifold Learning for Images , 2009, IPSJ Trans. Comput. Vis. Appl..

[35]  Daniel D. Lee,et al.  Grassmann discriminant analysis: a unifying view on subspace-based learning , 2008, ICML '08.

[36]  Yousef Saad,et al.  Enhanced Multilevel Manifold Learning , 2012 .

[37]  Serge J. Belongie,et al.  Learning to Traverse Image Manifolds , 2006, NIPS.

[38]  Rudolf Fleischer,et al.  Distance Approximating Dimension Reduction of Riemannian Manifolds , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[39]  Nakul Verma,et al.  Distance Preserving Embeddings for General n-Dimensional Manifolds , 2012, COLT.

[40]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[41]  Deli Zhao,et al.  Linear local tangent space alignment and application to face recognition , 2007, Neurocomputing.

[42]  Stergios B. Fotopoulos,et al.  All of Nonparametric Statistics , 2007, Technometrics.

[43]  Alon Zakai,et al.  Manifold Learning: The Price of Normalization , 2008, J. Mach. Learn. Res..

[44]  Alexander M. Bronstein,et al.  Nonlinear Dimensionality Reduction by Topologically Constrained Isometric Embedding , 2010, International Journal of Computer Vision.

[45]  Snigdhansu Chatterjee,et al.  Procrustes Problems , 2005, Technometrics.

[46]  Anders Brun,et al.  Manifold learning and representations for image analysis and visualization , 2006 .

[47]  A. Bernstein Data-based Manifold Reconstruction via Tangent Bundle Manifold Learning , 2014 .

[48]  Serge J. Belongie,et al.  Non-isometric manifold learning: analysis and an algorithm , 2007, ICML '07.

[49]  Michel Verleysen,et al.  Quality assessment of dimensionality reduction: Rank-based criteria , 2009, Neurocomputing.

[50]  Yunqian Ma,et al.  Manifold Learning Theory and Applications , 2011 .

[51]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[52]  Yaacov Ritov,et al.  Local procrustes for manifold embedding: a measure of embedding quality and embedding algorithms , 2009, Machine Learning.

[53]  Daniel Rueckert,et al.  Manifold Learning for Medical Image Registration, Segmentation, and Classification , 2012 .

[54]  A. Singer,et al.  Vector diffusion maps and the connection Laplacian , 2011, Communications on pure and applied mathematics.

[55]  Hong Qiao,et al.  An improved local tangent space alignment method for manifold learning , 2011, Pattern Recognit. Lett..

[56]  X. Huo,et al.  A Survey of Manifold-Based Learning Methods , 2007 .

[57]  Jacinto C. Nascimento,et al.  Manifold Learning for Object Tracking with Multiple Motion Dynamics , 2010, ECCV.

[58]  Chao Chen,et al.  A Nonlinear Dimension Reduction Method with Both Distance and Neighborhood Preservation , 2013, KSEM.

[59]  Liwei Wang,et al.  Subspace distance analysis with application to adaptive Bayesian algorithm for face recognition , 2006, Pattern Recognit..

[60]  Lawrence K. Saul,et al.  Analysis and extension of spectral methods for nonlinear dimensionality reduction , 2005, ICML.

[61]  I. Holopainen Riemannian Geometry , 1927, Nature.

[62]  Hongbin Zha,et al.  Riemannian Manifold Learning , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Tony Jebara,et al.  Minimum Volume Embedding , 2007, AISTATS.

[64]  Kilian Q. Weinberger,et al.  Nonlinear Dimensionality Reduction by Semidefinite Programming and Kernel Matrix Factorization , 2005, AISTATS.

[65]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[66]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[67]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[68]  Ali Ghodsi,et al.  Conformal Mapping by Computationally Efficient Methods , 2010, AAAI.

[69]  X. Huo,et al.  Electricity Price Curve Modeling and Forecasting by Manifold Learning , 2008, IEEE Transactions on Power Systems.