An Analysis of the Convergence of Graph Laplacians

Existing approaches to analyzing the asymptotics of graph Laplacians typically assume a well-behaved kernel function with smoothness assumptions. We remove the smoothness assumption and generalize the analysis of graph Laplacians to include previously unstudied graphs including kNN graphs. We also introduce a kernel-free framework to analyze graph constructions with shrinking neighborhoods in general and apply it to analyze locally linear embedding (LLE). We also describe how, for a given limit operator, desirable properties such as a convergent spectrum and sparseness can be achieved by choosing the appropriate graph construction.

[1]  C. Quesenberry,et al.  A nonparametric estimate of a multivariate density function , 1965 .

[2]  W. Boothby.  An introduction to differentiable manifolds and Riemannian geometry , 1975 .

[3]  L. Devroye,et al.  The Strong Uniform Consistency of Nearest Neighbor Density Estimates. , 1977 .

[4]  Ian Abramson.  On Bandwidth Variation in Kernel Estimates-A Square Root Law , 1982 .

[5]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[6]  A. Shiryaev,et al.  Limit Theorems for Stochastic Processes , 1987 .

[7]  D. W. Scott,et al.  Variable Kernel Density Estimation , 1992 .

[8]  O. Kallenberg.  Foundations of Modern Probability , 2021, Probability Theory and Stochastic Modelling.

[9]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[10]  Santosh S. Vempala,et al.  On clusterings-good, bad and spectral , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[11]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[12]  Matthias Hein,et al.  Measure Based Regularization , 2003, NIPS.

[13]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[14]  Pietro Perona,et al.  Self-Tuning Spectral Clustering , 2004, NIPS.

[15]  Mikhail Belkin,et al.  Semi-Supervised Learning on Riemannian Manifolds , 2004, Machine Learning.

[16]  Bruno Pelletier.  Kernel density estimation on Riemannian manifolds , 2005 .

[17]  Ulrike von Luxburg,et al.  From Graphs to Manifolds - Weak and Strong Pointwise Consistency of Graph Laplacians , 2005, COLT.

[18]  Xiaojin Zhu,et al.  Semi-Supervised Learning Literature Survey , 2005 .

[19]  B. Nadler,et al.  Diffusion maps, spectral clustering and reaction coordinates of dynamical systems , 2005, math/0503445.

[20]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[21]  Mikhail Belkin,et al.  Convergence of Laplacian Eigenmaps , 2006, NIPS.

[22]  V. Koltchinskii,et al.  Empirical graph Laplacian approximation of Laplace–Beltrami operators: Large sample results , 2006, math/0612777.

[23]  A. Singer.  From graph to manifold Laplacian: The convergence rate , 2006 .

[24]  Ulrike von Luxburg,et al.  Graph Laplacians and their Convergence on Random Neighborhood Graphs , 2006, J. Mach. Learn. Res..

[25]  Mikhail Belkin,et al.  Towards a theoretical foundation for Laplacian-based manifold methods , 2005, J. Comput. Syst. Sci..

[26]  Mikhail Belkin,et al.  Consistency of spectral clustering , 2008, 0804.0678.

[27]  Ulrike von Luxburg,et al.  Influence of graph construction on graph-based clustering measures , 2008, NIPS.

[28]  Alexander G. Gray,et al.  Submanifold density estimation , 2009, NIPS.