Pre-image Problem in Manifold Learning and Dimensional Reduction Methods

Manifold learning and dimensional reduction methods provide a low dimensional embedding for a collection of training samples. These methods are based on the eigenvalue decomposition of the kernel matrix formed using the training samples. In [2] the embedding is extended to new test samples using the Nystrom approximation method. This paper addresses the pre-image problem for these methods, which is to find the mapping back from the embedding space to the input space for new test points. The relationship of these learning methods to kernel principal component analysis [6] and the connection of the out-of-sample problem to the pre-image problem [1] is used to provide the pre-image.

[1]  Allen Tannenbaum,et al.  Statistical shape analysis using kernel PCA , 2006, Electronic Imaging.

[2]  Michael E. Leventon,et al.  Statistical models in medical image analysis , 2000 .

[3]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[4]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[5]  D. DeCoste.  Visualizing Mercer Kernel feature spaces via kernelized locally-linear embeddings , 2001 .

[6]  Yoshua Bengio,et al.  Learning Eigenfunctions of Similarity: Linking Spectral Clustering and Kernel PCA , 2003 .

[7]  Guillermo Sapiro,et al.  Connecting the Out-of-Sample and Pre-Image Problems in Kernel Methods , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[9]  Gunnar Rätsch,et al.  Input space versus feature space in kernel-based methods , 1999, IEEE Trans. Neural Networks.

[10]  Steven W. Zucker,et al.  Diffusion Maps and Geometric Harmonics for Automatic Target Recognition (ATR). Volume 2. Appendices , 2007 .

[11]  Yogesh Rathi,et al.  Comparative Analysis of Kernel Methods for Statistical Shape Learning , 2006, CVAMIA.

[12]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[13]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[14]  Stan Z. Li,et al.  Manifold Learning and Applications in Recognition , 2005 .

[15]  Bernhard Schölkopf,et al.  A kernel view of the dimensionality reduction of manifolds , 2004, ICML.

[16]  Ivor W. Tsang,et al.  The pre-image problem in kernel methods , 2003, IEEE Transactions on Neural Networks.

[17]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[18]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[19]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.