Metric Learning on Manifolds

In recent years, manifold learning has become increasingly popular as a tool for performing nonlinear dimensionality reduction. This has led to the development of numerous algorithms of varying degrees of complexity that aim to recover manifold geometry using either local or global features of the data. Building on the Laplacian Eigenmap framework, we propose a new paradigm that oers a guarantee, under reasonable assumptions, that any manifold learning algorithm will preserve the geometry of a data set. Our approach is based on augmenting the output of embedding algorithms with geometric information embodied in the Riemannian metric of the manifold. The Riemannian metric allows us to compute geometric quantities (such as angle, length, or volume) for any coordinate system or embedding of the manifold. This geometric faithfulness, which is not guaranteed for most algorithms, allows us to dene geometric measurements that are independent of the algorithm used, and hence move seamlessly from one algorithm to another. In this work, we provide an algorithm for estimating the Riemannian metric from data, consider its consistency, and demonstrate the advantages of our approach in a variety of examples.

[1]  J. Nash The imbedding problem for Riemannian manifolds , 1956 .

[2]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[3]  S. Rosenberg The Laplacian on a Riemannian Manifold: The Laplacian on a Riemannian Manifold , 1997 .

[4]  John M. Lee Riemannian Manifolds: An Introduction to Curvature , 1997 .

[5]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[6]  D. Harville Matrix Algebra From a Statistician's Perspective , 1998 .

[7]  Christopher J. C. Burges,et al.  Geometry and invariance in kernel based methods , 1999 .

[8]  Vin de Silva,et al.  Graph approximations to geodesics on embedded manifolds , 2000 .

[9]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[10]  John M. Lee Introduction to Smooth Manifolds , 2002 .

[11]  Matthew Brand,et al.  Charting a Manifold , 2002, NIPS.

[12]  Hongyuan Zha,et al.  Isometric Embedding and Continuum ISOMAP , 2003, ICML.

[13]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[14]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[15]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[16]  张振跃,et al.  Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment , 2004 .

[17]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[18]  Ronald R. Coifman,et al.  Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators , 2005, NIPS.

[19]  Lawrence K. Saul,et al.  Analysis and extension of spectral methods for nonlinear dimensionality reduction , 2005, ICML.

[20]  B. Nadler,et al.  Diffusion maps, spectral clustering and reaction coordinates of dynamical systems , 2005, math/0503445.

[21]  David L. Donoho,et al.  Image Manifolds which are Isometric to Euclidean Space , 2005, Journal of Mathematical Imaging and Vision.

[22]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[23]  V. Koltchinskii,et al.  Empirical graph Laplacian approximation of Laplace–Beltrami operators: Large sample results , 2006, math/0612777.

[24]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[25]  A. Singer From graph to manifold Laplacian: The convergence rate , 2006 .

[26]  Ulrike von Luxburg,et al.  Graph Laplacians and their Convergence on Random Neighborhood Graphs , 2006, J. Mach. Learn. Res..

[27]  Miguel Á. Carreira-Perpiñán,et al.  People Tracking with the Laplacian Eigenmaps Latent Variable Model , 2007, NIPS.

[28]  Alon Zakai,et al.  Manifold Learning: The Price of Normalization , 2008, J. Mach. Learn. Res..

[29]  Mikhail Belkin,et al.  Consistency of spectral clustering , 2008, 0804.0678.

[30]  Hongbin Zha,et al.  Riemannian Manifold Learning , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  William B. March,et al.  Linear-time Algorithms for Pairwise Statistical Problems , 2009, NIPS.

[32]  Mikhail Belkin,et al.  Constructing Laplace operator from point clouds in Rd , 2009, SODA.

[33]  Yaacov Ritov,et al.  Local procrustes for manifold embedding: a measure of embedding quality and embedding algorithms , 2009, Machine Learning.

[34]  Ling Huang,et al.  An Analysis of the Convergence of Graph Laplacians , 2010, ICML.

[35]  Raviv Raich,et al.  Isometric Correction for Manifold Learning , 2010, AAAI Fall Symposium: Manifold Learning and Its Applications.

[36]  John P. Snyder,et al.  Map Projections: A Working Manual , 2012 .