Metric Learning and Manifolds: Preserving the Intrinsic Geometry

A variety of algorithms exist for performing non-linear dimension reduction, but these algorithms do not preserve the original geometry of the data except in special cases. In general, in the low-dimensional representations obtained, distances are distorted, as well as angles, areas, etc. This paper proposes a generic method to estimate the distortion incurred at each point of an embedding, and subsequently to correct distances and other intrinsic geometric quantities back to their original values (up to sampling noise). Our approach is based on augmenting the output of an embedding algorithm with geometric information embodied in the Riemannian metric of the manifold. The Riemannian metric allows one to compute geometric quantities (such as angle, length, or volume) for any coordinate system or embedding of the manifold. In this work, we provide an algorithm for estimating the Riemannian metric from data, consider its consistency, and demonstrate the uses of our approach in a variety of examples.

[1]  Hongbin Zha,et al.  Riemannian Manifold Learning , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  John M. Lee.  Riemannian Manifolds: An Introduction to Curvature , 1997 .

[3]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, CVPR.

[4]  J. Nash.  The imbedding problem for Riemannian manifolds , 1956 .

[5]  Byron M. Yu,et al.  Dimensionality reduction for large-scale neural recordings , 2014, Nature Neuroscience.

[6]  V. Koltchinskii,et al.  Empirical graph Laplacian approximation of Laplace–Beltrami operators: Large sample results , 2006, math/0612777.

[7]  Mikhail Belkin,et al.  Consistency of spectral clustering , 2008, 0804.0678.

[8]  Hongyuan Zha,et al.  Isometric Embedding and Continuum ISOMAP , 2003, ICML.

[9]  David L. Donoho,et al.  Image Manifolds which are Isometric to Euclidean Space , 2005, Journal of Mathematical Imaging and Vision.

[10]  D. Harville.  Matrix Algebra From a Statistician's Perspective , 1998 .

[11]  Ulrike von Luxburg,et al.  Graph Laplacians and their Convergence on Random Neighborhood Graphs , 2006, J. Mach. Learn. Res..

[12]  William B. March,et al.  Linear-time Algorithms for Pairwise Statistical Problems , 2009, NIPS.

[13]  Klaus-Robert Müller,et al.  Machine learning of accurate energy-conserving molecular force fields , 2016, Science Advances.

[14]  Mikhail Belkin,et al.  Constructing Laplace operator from point clouds in Rd , 2009, SODA.

[15]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[16]  Nakul Verma,et al.  Distance Preserving Embeddings for General n-Dimensional Manifolds , 2012, COLT.

[17]  S. Rosenberg.  The Laplacian on a Riemannian Manifold: The Construction of the Heat Kernel , 1997 .

[18]  Yaacov Ritov,et al.  Local procrustes for manifold embedding: a measure of embedding quality and embedding algorithms , 2009, Machine Learning.

[19]  Ling Huang,et al.  An Analysis of the Convergence of Graph Laplacians , 2010, ICML.

[20]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[21]  Marina Meila,et al.  Megaman: Scalable Manifold Learning in Python , 2016, J. Mach. Learn. Res..

[22]  Michele Parrinello,et al.  Simplifying the representation of complex free-energy landscapes using sketch-map , 2011, Proceedings of the National Academy of Sciences.

[23]  Miguel Á. Carreira-Perpiñán,et al.  People Tracking with the Laplacian Eigenmaps Latent Variable Model , 2007, NIPS.

[24]  John M. Lee.  Introduction to Smooth Manifolds , 2002 .

[25]  A. Singer.  From graph to manifold Laplacian: The convergence rate , 2006 .

[26]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[27]  Alon Zakai,et al.  Manifold Learning: The Price of Normalization , 2008, J. Mach. Learn. Res..

[28]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[29]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[30]  M. Stopfer,et al.  Encoding a temporally structured stimulus with a temporally structured neural representation , 2005, Nature Neuroscience.

[31]  Raviv Raich,et al.  Isometric Correction for Manifold Learning , 2010, AAAI Fall Symposium: Manifold Learning and Its Applications.

[32]  Andrew J. Connolly,et al.  CLASSIFICATION OF STELLAR SPECTRA WITH LOCAL LINEAR EMBEDDING , 2011 .