Connecting dots: from local covariance to empirical intrinsic geometry and locally linear embedding

Local covariance structure under the manifold setup has been widely applied in the machine learning society. Based on the established theoretical results, we provide an extensive study of two relevant manifold learning algorithms, empirical intrinsic geometry (EIG) and the locally linear embedding (LLE) under the manifold setup. Particularly, we show that without an accurate dimension estimation, the geodesic distance estimation by EIG might be corrupted. Furthermore, we show that by taking the local covariance matrix into account, we can more accurately estimate the local geodesic distance. When understanding LLE based on the local covariance structure, its intimate relationship with the curvature suggests a variation of LLE depending on the "truncation scheme". We provide a theoretical analysis of the variation.

[1]  Hau-Tieng Wu,et al.  Local Linear Regression on Manifolds and Its Geometric Interpretation , 2012, 1201.0327.

[2]  Robert D. Nowak,et al.  Multi-Manifold Semi-Supervised Learning , 2009, AISTATS.

[3]  Matthew Brand,et al.  Charting a Manifold , 2002, NIPS.

[4]  Hau-Tieng Wu,et al.  Think globally, fit locally under the manifold setup: Asymptotic analysis of locally linear embedding , 2017, The Annals of Statistics.

[5]  Hau-Tieng Wu,et al.  Assess Sleep Stage by Modern Signal Processing Techniques , 2014, IEEE Transactions on Biomedical Engineering.

[6]  R. Coifman,et al.  Empirical Intrinsic Modeling of Signals and Information Geometry , 2012 .

[7]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[8]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[9]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[10]  A. Averbuch,et al.  Patch-to-tensor embedding , 2012 .

[11]  Ronen Talmon,et al.  Empirical intrinsic geometry for nonlinear modeling and time series filtering , 2013, Proceedings of the National Academy of Sciences.

[12]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[13]  Pascal Frossard,et al.  Tangent space estimation for smooth embeddings of Riemannian manifolds , 2012 .

[14]  Ronen Talmon,et al.  Local Canonical Correlation Analysis for Nonlinear Common Variables Discovery , 2016, IEEE Transactions on Signal Processing.

[15]  Israel Cohen,et al.  Graph-Based Supervised Automatic Target Detection , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Gérard G. Medioni,et al.  Robust Multiple Manifolds Structure Learning , 2012, ICML 2012.

[17]  Nanda Kambhatla,et al.  Dimension Reduction by Local Principal Component Analysis , 1997, Neural Computation.

[18]  Yaacov Ritov,et al.  LDR-LLE: LLE with Low-Dimensional Neighborhood Representation , 2008, ISVC.

[19]  Ronen Talmon,et al.  Affective response to architecture – investigating human reaction to spaces with different geometry , 2017 .

[20]  L. Rosasco,et al.  Multiscale geometric methods for data sets I: Multiscale SVD, noise and curvature , 2017 .

[21]  A. Singer,et al.  Vector diffusion maps and the connection Laplacian , 2011, Communications on pure and applied mathematics.

[22]  Achi Brandt,et al.  Fast multiscale clustering and manifold identification , 2006, Pattern Recognit..

[23]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[24]  R. Coifman,et al.  Non-linear independent component analysis with diffusion maps , 2008 .

[25]  H. Weizsäcker,et al.  Chernoff's Theorem and Discrete Time Approximations of Brownian Motion on Manifolds , 2004, math/0409155.

[26]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[27]  Edwin R. Hancock,et al.  Clustering and Embedding Using Commute Times , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Daniel N. Kaslovsky,et al.  Non-Asymptotic Analysis of Tangent Space Perturbation , 2011 .

[29]  Hau-Tieng Wu,et al.  Diffuse to fuse EEG spectra - Intrinsic geometry of sleep dynamics for classification , 2018, Biomed. Signal Process. Control..

[30]  Sergio A. Velastin,et al.  Local Fisher Discriminant Analysis for Pedestrian Re-identification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.