Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment

In this paper we present a new algorithm for manifold learning and nonlinear dimension reduction. Based on a set of unorganized data points sampled with noise from the manifold, we represent the local geometry of the manifold using tangent spaces learned by fitting an affine subspace in a neighborhood of each data point. Those tangent spaces are aligned to give the internal global coordinates of the data points with respect to the underlying manifold by way of a partial eigendecomposition of the neighborhood connection matrix. We present a careful error analysis of our algorithm and show that the reconstruction errors are of second-order accuracy. Numerical experiments including 64-by-64 pixel face images are given to illustrate our algorithm.

[1]  Daniel Freedman,et al.  Efficient Simplicial Reconstructions of Manifolds from Their Samples , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  J. R. Koehler,et al.  Modern Applied Statistics with S-Plus. , 1996 .

[3]  Henry W. Altland,et al.  Applied Functional Data Analysis , 2003, Technometrics.

[4]  Calvin L. Williams,et al.  Modern Applied Statistics with S-Plus , 1997 .

[5]  L. Saul,et al.  Think globally, fit locally: unsupervised l earning of non-linear manifolds , 2002 .

[6]  Geoffrey E. Hinton,et al.  Stochastic Neighbor Embedding , 2002, NIPS.

[7]  Thomas Martinetz,et al.  Topology representing networks , 1994, Neural Networks.

[8]  M. Naderi.  Think globally... , 2004, HIV prevention plus!.

[9]  Geoffrey E. Hinton,et al.  Global Coordination of Local Linear Models , 2001, NIPS.

[10]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[11]  Pietro Perona,et al.  Grouping and dimensionality reduction by locally linear embedding , 2001, NIPS.

[12]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[13]  Maja J. Mataric,et al.  Deriving action and behavior primitives from human motion data , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[15]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[16]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.