LDLE: Low Distortion Local Eigenmaps

We present Low Distortion Local Eigenmaps (LDLE), a manifold learning technique which constructs a set of low distortion local views of a dataset in lower dimension and registers them to obtain a global embedding. The local views are constructed using the global eigenvectors of the graph Laplacian and are registered using Procrustes analysis. The choice of these eigenvectors may vary across the regions. In contrast to existing techniques, LDLE can embed closed and non-orientable manifolds into their intrinsic dimension by tearing them apart. It also provides gluing instruction on the boundary of the torn embedding to help identify the topology of the original manifold. Our experimental results will show that LDLE largely preserved distances up to a constant scale while other techniques produced higher distortion. We also demonstrate that LDLE produces high quality embeddings even when the data is noisy or sparse.

[1]  Xiuyuan Cheng,et al.  Eigen-convergence of Gaussian kernelized graph Laplacian by manifold heat interpolation , 2021 .

[2]  Mikhail Belkin,et al.  Towards a Theoretical Foundation for Laplacian-Based Manifold Methods , 2005, COLT.

[3]  S. Steinerberger.  Lower Bounds on Nodal Sets of Eigenfunctions via the Heat Flow , 2013, 1301.3371.

[4]  Hau-Tieng Wu,et al.  Convergence of Graph Laplacian with kNN Self-tuned Kernels , 2020, Information and Inference: A Journal of the IMA.

[5]  Stefan Steinerberger,et al.  The Geometry of Nodal Sets and Outlier Detection , 2017, 1706.01362.

[6]  Alexander Cloninger,et al.  Diffusion Nets , 2015, Applied and Computational Harmonic Analysis.

[7]  Ronald R. Coifman,et al.  Local conformal autoencoder for standardized data coordinates , 2020, Proceedings of the National Academy of Sciences.

[8]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[9]  Naoki Saito,et al.  How Can We Naturally Order and Organize Graph Laplacian Eigenvectors? , 2018, 2018 IEEE Statistical Signal Processing Workshop (SSP).

[10]  Stefan Steinerberger.  On the spectral resolution of products of Laplacian eigenfunctions , 2019, Journal of Spectral Theory.

[11]  Carmeline J. Dsilva,et al.  Parsimonious Representation of Nonlinear Dynamical Systems Through Manifold Learning: A Chemotaxis Case Study , 2015, 1505.06118.

[12]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[13]  Ulrike von Luxburg,et al.  Graph Laplacians and their Convergence on Random Neighborhood Graphs , 2006, J. Mach. Learn. Res..

[14]  Leland McInnes,et al.  UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction , 2018, ArXiv.

[15]  Mikhail Belkin,et al.  Towards a theoretical foundation for Laplacian-based manifold methods , 2005, J. Comput. Syst. Sci..

[16]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[17]  Xiuyuan Cheng,et al.  Spectral Embedding Norm: Looking Deep into the Spectrum of the Graph Laplacian , 2018, SIAM J. Imaging Sci..

[18]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[19]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Ronald R. Coifman,et al.  Automated cellular structure extraction in biological images with applications to calcium imaging data , 2018, bioRxiv.

[21]  Jorge Cadima,et al.  Principal component analysis: a review and recent developments , 2016, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[22]  D. Dunson,et al.  Geodesic Distance Estimation with Spherelets , 2019, 1907.00296.

[23]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[24]  Amit Singer,et al.  Product Manifold Learning , 2020, AISTATS.

[25]  Israel Cohen,et al.  Multiscale Anomaly Detection Using Diffusion Maps , 2013, IEEE Journal of Selected Topics in Signal Processing.

[26]  Marina Meila,et al.  Selecting the independent coordinates of manifolds with large aspect ratios , 2019, NeurIPS.

[27]  J. Berge,et al.  Orthogonal procrustes rotation for two or more matrices , 1977 .

[28]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[29]  Matthias Hein,et al.  Error Estimates for Spectral Convergence of the Graph Laplacian on Random Geometric Graphs Toward the Laplace–Beltrami Operator , 2018, Found. Comput. Math..

[30]  Hau-Tieng Wu,et al.  Local Linear Regression on Manifolds and Its Geometric Interpretation , 2012, 1201.0327.

[31]  P. Schönemann,et al.  A generalized solution of the orthogonal procrustes problem , 1966 .

[32]  P. Alam.  ‘G’ , 2021, Composites Engineering: An A–Z Guide.

[33]  Alexander Cloninger,et al.  On the Dual Geometry of Laplacian Eigenfunctions , 2018, Exp. Math..

[34]  M. Ehler.  Applied and Computational Harmonic Analysis , 2008 .

[35]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[36]  Timothy D. Sauer,et al.  Density estimation on manifolds with boundary , 2015, Comput. Stat. Data Anal..

[37]  J. Gower.  Generalized procrustes analysis , 1975 .

[38]  Yochai Blau,et al.  Non-redundant Spectral Dimensionality Reduction , 2016, ECML/PKDD.

[39]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[40]  Ulrike von Luxburg,et al.  Measures of distortion for machine learning , 2018, NeurIPS.

[41]  Roy R. Lederman,et al.  Learning the geometry of common latent variables using alternating-diffusion , 2015 .

[42]  J. Hintze,et al.  Violin plots : A box plot-density trace synergism , 1998 .

[43]  D. Kobak,et al.  Initialization is critical for preserving global data structure in both t-SNE and UMAP , 2021, Nature Biotechnology.

[44]  Fabio Crosilla,et al.  Use of generalised Procrustes analysis for the photogrammetric block adjustment by independent models , 2002 .

[45]  Wojciech Czaja,et al.  Eigenvector localization on data-dependent graphs , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[46]  P. Bickel,et al.  Regression on manifolds: Estimation of the exterior derivative , 2011, 1103.1457.

[47]  A. Singer,et al.  Orientability and Diffusion Maps. , 2011, Applied and computational harmonic analysis.

[48]  M. Maggioni,et al.  Universal Local Parametrizations via Heat Kernels and Eigenfunctions of the Laplacian , 2007, 0709.1975.

[49]  Nicolas Garcia Trillos,et al.  Large sample spectral analysis of graph-based multi-manifold clustering , 2021, ArXiv.