Laplacian Eigenmaps for Dimensionality Reduction and Data Representation

One of the central problems in machine learning and pattern recognition is to develop appropriate representations for complex data. We consider the problem of constructing a representation for data lying on a low-dimensional manifold embedded in a high-dimensional space. Drawing on the correspondence between the graph Laplacian, the Laplace Beltrami operator on the manifold, and the connections to the heat equation, we propose a geometrically motivated algorithm for representing the high-dimensional data. The algorithm provides a computationally efficient approach to nonlinear dimensionality reduction that has locality-preserving properties and a natural connection to clustering. Some potential applications and illustrative examples are discussed.

[1]  J. Nash.  C 1 Isometric Imbeddings , 1954 .

[2]  J. Nash.  The imbedding problem for Riemannian manifolds , 1956 .

[3]  Horst D. Simon,et al.  Partitioning of unstructured problems for parallel processing , 1991 .

[4]  Brian L. Mark,et al.  An efficient eigenvector approach for finding netlist partitions , 1992, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[5]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[6]  F. Chung.  Spectral Graph Theory, Regional Conference Series in Math. , 1997 .

[7]  S. Rosenberg.  The Laplacian on a Riemannian Manifold: The Construction of the Heat Kernel , 1997 .

[8]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[10]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[11]  Vin de Silva,et al.  Graph approximations to geodesics on embedded manifolds , 2000 .

[12]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[13]  F. Chung,et al.  Higher eigenvalues and isoperimetric inequalities on Riemannian manifolds and graphs , 2000 .

[14]  Piotr Indyk.  Dimensionality reduction techniques for proximity problems , 2000, SODA '00.

[15]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[16]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[17]  H. Sebastian Seung,et al.  The Manifold Ways of Perception , 2000, Science.

[18]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[19]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[20]  John D. Lafferty,et al.  Diffusion Kernels on Graphs and Other Discrete Input Spaces , 2002, ICML.

[21]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[22]  Vin de Silva,et al.  Unsupervised Learning of Curved Manifolds , 2003 .

[23]  Edwin R. Hancock,et al.  A Spectral Approach to Learning Structural Variations in Graphs , 2003, ICVS.

[24]  A. Gelperin,et al.  Volatile metabolic monitoring of glycemic status in diabetes using electronic olfaction. , 2004, Diabetes technology & therapeutics.

[25]  R. Coifman,et al.  Diffusion Wavelets , 2004 .

[26]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[27]  Nicolas Le Roux,et al.  Learning Eigenfunctions Links Spectral Embedding and Kernel PCA , 2004, Neural Computation.

[28]  Edwin R. Hancock,et al.  Pattern Vectors from Algebraic Graph Theory , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  H. Zha,et al.  Principal manifolds and nonlinear dimensionality reduction via tangent space alignment , 2004, SIAM J. Sci. Comput..

[30]  Helge J. Ritter,et al.  Principal surfaces from unsupervised kernel regression , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Mauro Maggioni,et al.  Geometric diffusions for the analysis of data from sensor networks , 2005, Current Opinion in Neurobiology.

[32]  I. Kevrekidis,et al.  Optical imaging and control of genetically designated neurons in functioning circuits. , 2005, Annual review of neuroscience.

[33]  Fabio Boschetti,et al.  Dimensionality reduction and visualisation of geoscientific images via locally linear embedding 1 , 2004 .

[34]  Geoffrey E. Hinton,et al.  Improving dimensionality reduction with spectral gradient descent , 2005, Neural Networks.

[35]  B. Nadler,et al.  Diffusion maps, spectral clustering and reaction coordinates of dynamical systems , 2005, math/0503445.

[36]  Yap-Peng Tan,et al.  Shot boundary classification by temporal pattern discovery from Laplacian eigenmap , 2005 .