Embedding new data points for manifold learning via coordinate propagation

In recent years, a series of manifold learning algorithms have been proposed for nonlinear dimensionality reduction. Most of them can run in a batch mode for a set of given data points, but lack a mechanism to deal with new data points. Here we propose an extension approach, i.e., mapping new data points into the previously learned manifold. The core idea of our approach is to propagate the known coordinates to each of the new data points. We first formulate this task as a quadratic programming, and then develop an iterative algorithm for coordinate propagation. Tangent space projection and smooth splines are used to yield an initial coordinate for each new data point, according to their local geometrical relations. Experimental results and applications to camera direction estimation and face pose estimation illustrate the validity of our approach.

[1]  Daniel D. Lee,et al.  Semisupervised alignment of manifolds , 2005, AISTATS.

[2]  Lawrence K. Saul,et al.  Analysis and extension of spectral methods for nonlinear dimensionality reduction , 2005, ICML.

[3]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[4]  Hanqing Lu,et al.  A semi-supervised framework for mapping data to the intrinsic manifold , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[5]  Ke Lu,et al.  Locality pursuit embedding , 2004, Pattern Recognition.

[6]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[7]  Carl-Fredrik Westin,et al.  Coloring of DT-MRI Fiber Traces Using Laplacian Eigenmaps , 2003, EUROCAST.

[8]  Anil K. Jain,et al.  Incremental nonlinear dimensionality reduction by manifold learning , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Christopher M. Bishop,et al.  GTM: The Generative Topographic Mapping , 1998, Neural Computation.

[10]  Maja J. Mataric,et al.  A spatio-temporal extension to Isomap nonlinear dimension reduction , 2004, ICML.

[11]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[12]  Kilian Q. Weinberger,et al.  Learning a kernel matrix for nonlinear dimensionality reduction , 2004, ICML.

[13]  Lorenzo Rosasco,et al.  Dimensionality reduction and generalization , 2007, ICML '07.

[14]  D. Yeung,et al.  Super-resolution through neighbor embedding , 2004, CVPR 2004.

[15]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[16]  Venu Govindaraju,et al.  Generalized regression model for sequence matching and clustering , 2007, Knowledge and Information Systems.

[17]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[18]  R. Tibshirani Principal curves revisited , 1992 .

[19]  Jiawei Han,et al.  Efficient Kernel Discriminant Analysis via Spectral Regression , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[20]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[21]  Bianca Zadrozny,et al.  Ranking-based evaluation of regression models , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[22]  Feiping Nie,et al.  Spline Embedding for Nonlinear Dimensionality Reduction , 2006, ECML.

[23]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[24]  Xin Yang,et al.  Semi-supervised nonlinear dimensionality reduction , 2006, ICML.

[25]  Daoqiang Zhang,et al.  Semi-Supervised Dimensionality Reduction ∗ , 2007 .

[26]  Matti Pietikäinen,et al.  Unsupervised learning using locally linear embedding: experiments with face pose analysis , 2002, Object recognition supported by user interaction for service robots.

[27]  Dimitrios Gunopulos,et al.  Non-linear dimensionality reduction techniques for classification and visualization , 2002, KDD.

[28]  Shuicheng Yan,et al.  Graph embedding: a general framework for dimensionality reduction , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[29]  I. Jolliffe Principal Component Analysis , 2002 .

[30]  Jungho Yoon,et al.  Spectral Approximation Orders of Radial Basis Function Interpolation on the Sobolev Space , 2001, SIAM J. Math. Anal..

[31]  Gene H. Golub,et al.  Matrix computations , 1983 .

[32]  R. C. Williamson,et al.  Regularized principal manifolds , 2001 .

[33]  Pierre-Antoine Absil,et al.  Principal Manifolds for Data Visualization and Dimension Reduction , 2007 .

[34]  Matthew Brand,et al.  Charting a Manifold , 2002, NIPS.

[35]  Jian Guan,et al.  Color by linear neighborhood embedding , 2005, IEEE International Conference on Image Processing 2005.

[36]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[37]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[38]  Miguel Á. Carreira-Perpiñán,et al.  The Laplacian Eigenmaps Latent Variable Model , 2007, AISTATS.

[39]  D. Donoho,et al.  Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[41]  Ming-Hsuan Yang,et al.  Face recognition using extended isomap , 2002, Proceedings. International Conference on Image Processing.

[42]  Melanie Hilario,et al.  Stability of feature selection algorithms , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[43]  Adam Krzyzak,et al.  Learning and Design of Principal Curves , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[44]  Serge J. Belongie,et al.  Learning to Traverse Image Manifolds , 2006, NIPS.

[45]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[46]  Matti Pietikäinen,et al.  Incremental locally linear embedding , 2005, Pattern Recognit..

[47]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[48]  Jean Duchon,et al.  Splines minimizing rotation-invariant semi-norms in Sobolev spaces , 1976, Constructive Theory of Functions of Several Variables.

[49]  张振跃,et al.  Principal Manifolds and Nonlinear Dimensionality Reduction via Tangent Space Alignment , 2004 .

[50]  Anil K. Jain,et al.  Artificial neural networks for feature extraction and multivariate data projection , 1995, IEEE Trans. Neural Networks.

[51]  Olli Silven,et al.  Comparison of dimensionality reduction methods for wood surface inspection , 2003, International Conference on Quality Control by Artificial Vision.