Analysis of alignment algorithms with mixed dimensions for dimensionality reduction

SUMMARY We consider an alignment algorithm for reconstructing global coordinates of a given data set from coordinates constructed for data points in small local neighborhoods through computing a spectral subspace of an alignment matrix. We show that, under certain conditions, the null space of the alignment matrix recovers global coordinates even when local point sets have different dimensions. This result generalizes a previous analysis to allow alignment of local coordinates of mixed dimensions. We also extend this result to the setting of a semi-supervised learning problem, and we present several examples to illustrate our results. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Hongyuan Zha,et al.  Analysis of an alignment algorithm for nonlinear dimensionality reduction , 2007 .

[3]  Yee Whye Teh,et al.  Automatic Alignment of Local Representations , 2002, NIPS.

[4]  Mikhail Belkin,et al.  Learning speaker normalization using semisupervised manifold alignment , 2010, INTERSPEECH.

[5]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[6]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[7]  Q. Ye,et al.  Eigenvalue bounds for an alignment matrix in manifold learning , 2012 .

[8]  Hongyuan Zha,et al.  Spectral analysis of alignment in manifold learning , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[9]  Vin de Silva,et al.  Reduction A Global Geometric Framework for Nonlinear Dimensionality , 2011 .

[10]  Sridhar Mahadevan,et al.  Manifold alignment using Procrustes analysis , 2008, ICML '08.

[11]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[12]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[13]  Xiaoming Huo,et al.  Matrix perturbation analysis of local tangent space alignment , 2009 .

[14]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[15]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[16]  Ren-Cang Li,et al.  Eigenvalues of an alignment matrix in nonlinear manifold learning , 2007 .

[17]  Daniel D. Lee,et al.  Semisupervised alignment of manifolds , 2005, AISTATS.

[18]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[19]  Kilian Q. Weinberger,et al.  Spectral Methods for Dimensionality Reduction , 2006, Semi-Supervised Learning.