Variational Limits of k-NN Graph-Based Functionals on Data Clouds

This paper studies the large sample asymptotics of data analysis procedures based on the optimization of functionals defined on $k$-NN graphs on point clouds. The paper is framed in the context of minimization of balanced cut functionals, but our techniques, ideas and results can be adapted to other functionals of relevance. We rigorously show that provided the number of neighbors in the graph $k:=k_n$ scales with the number of points in the cloud as $n \gg k_n \gg \log(n)$, then with probability one, the solution to the graph cut optimization problem converges towards the solution of an analogue variational problem at the continuum level.

[1]  Augusto C. Ponce,et al.  A new approach to Sobolev spaces and connections to $\mathbf\Gamma$-convergence , 2004 .

[2]  Nicolas Garcia Trillos,et al.  A new analytical approach to consistency and overfitting in regularized empirical risk minimization , 2016, European Journal of Applied Mathematics.

[3]  D. Slepčev,et al.  Continuum Limit of Total Variation on Point Clouds , 2016 .

[4]  A. Singer.  From graph to manifold Laplacian: The convergence rate , 2006 .

[5]  Annalisa Baldi,et al.  WEIGHTED BV FUNCTIONS , 2001 .

[6]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[7]  Mikhail Belkin,et al.  Convergence of Laplacian Eigenmaps , 2006, NIPS.

[8]  Pierre Pudlo,et al.  The Normalized Graph Cut and Cheeger Constant: From Discrete to Continuous , 2010, Advances in Applied Probability.

[9]  Martin J. Wainwright,et al.  Asymptotic behavior of ℓp-based Laplacian regularization in semi-supervised learning , 2016, ArXiv.

[10]  Dejan Slepcev,et al.  A variational approach to the consistency of spectral clustering , 2015, Applied and Computational Harmonic Analysis.

[11]  Xavier Bresson,et al.  Consistency of Cheeger and Ratio Graph Cuts , 2014, J. Mach. Learn. Res..

[12]  Ulrike von Luxburg,et al.  From Graphs to Manifolds - Weak and Strong Pointwise Consistency of Graph Laplacians , 2005, COLT.

[13]  Matthias Hein,et al.  An Inverse Power Method for Nonlinear Eigenproblems with Applications in 1-Spectral Clustering and Sparse PCA , 2010, NIPS.

[14]  G. Leoni.  A First Course in Sobolev Spaces , 2009 .

[15]  Mikhail Belkin,et al.  Towards a theoretical foundation for Laplacian-based manifold methods , 2005, J. Comput. Syst. Sci..

[16]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[17]  Jeff Calder,et al.  The game theoretic p-Laplacian and semi-supervised learning with few labels , 2017, Nonlinearity.

[18]  W. Marsden.  I and J , 2012 .

[19]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[20]  Nicolás García Trillos,et al.  On the rate of convergence of empirical measures in $\infty$-transportation distance , 2014, 1407.1157.

[21]  Dejan Slepcev,et al.  Analysis of $p$-Laplacian Regularization in Semi-Supervised Learning , 2017, SIAM J. Math. Anal..

[22]  Xavier Bresson,et al.  Multiclass Total Variation Clustering , 2013, NIPS.

[23]  Matthias Hein,et al.  Beyond Spectral Clustering - Tight Relaxations of Balanced Graph Cuts , 2011, NIPS.

[24]  Zachary T. Kaplan,et al.  On the Consistency of Graph-based Bayesian Learning and the Scalability of Sampling Algorithms , 2017, ArXiv.

[25]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Matthias Hein,et al.  Constrained 1-Spectral Clustering , 2012, AISTATS.

[27]  V. Koltchinskii,et al.  Empirical graph Laplacian approximation of Laplace–Beltrami operators: Large sample results , 2006, math/0612777.

[28]  P. Grisvard.  Elliptic Problems in Nonsmooth Domains , 1985 .

[29]  Nicolás García Trillos,et al.  Continuum Limits of Posteriors in Graph Bayesian Inverse Problems , 2017, SIAM J. Math. Anal..

[30]  Amit Singer,et al.  Spectral Convergence of the connection Laplacian from random samples , 2013, 1306.1587.

[31]  Francesco Maggi,et al.  Sets of Finite Perimeter and Geometric Variational Problems: SETS OF FINITE PERIMETER , 2012 .

[32]  Mikhail Belkin,et al.  Consistency of spectral clustering , 2008, 0804.0678.

[33]  Ling Huang,et al.  An Analysis of the Convergence of Graph Laplacians , 2010, ICML.

[34]  Andrew M. Stuart,et al.  Uncertainty Quantification in the Classification of High Dimensional Data , 2017, ArXiv.

[35]  Xu Wang,et al.  Spectral Convergence Rate of Graph Laplacian , 2015, 1510.08110.

[36]  G. D. Maso,et al.  An Introduction to-convergence , 1993 .