论文信息 - Embedding via clustering: using spectral information to guide dimensionality reduction

Embedding via clustering: using spectral information to guide dimensionality reduction

We describe an approach to improve iterative dimensionality reduction methods by using information contained in the leading eigenvectors of a data affinity matrix. Using an insight from the area of spectral clustering, we suggest modifying the gradient of an iterative method, so that latent space elements belonging to the same cluster are encouraged to move in similar directions during optimization. We also describe way to achieve this without actually having to explicitly perform an eigendecomposition. Preliminary experiments show that our approach makes it possible to speed up iterative methods and helps them to find better local minima of their objective function.

Geoffrey E. Hinton | R. Memisevic | G. Hinton

[1] Bernhard Schölkopf,et al. Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[2] Yair Weiss,et al. Segmentation using eigenvectors: a unifying view , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[3] J. Tenenbaum,et al. A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[4] S T Roweis,et al. Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[5] Jianbo Shi,et al. A Random Walks View of Spectral Segmentation , 2001, AISTATS.

[6] Michael I. Jordan,et al. On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[7] Geoffrey E. Hinton,et al. Stochastic Neighbor Embedding , 2002, NIPS.

[8] Mikhail Belkin,et al. Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[9] Neil D. Lawrence,et al. Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data , 2003, NIPS.

[10] Geoffrey E. Hinton,et al. Multiple Relational Embedding , 2004, NIPS.

[11] Helge J. Ritter,et al. Principal surfaces from unsupervised kernel regression , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.