Relative concentration bounds for the spectrum of kernel matrices

In this paper we study some concentration properties of the kernel matrix associated with a kernel function. More specifically, we derive concentration inequalities for the spectrum of a kernel matrix, quantifying its deviation with respect to the spectrum of an associated integral operator. The main difference with most results in the literature is that we do not assume the positive definiteness of the kernel. Instead, we introduce Sobolev-type hypotheses on the regularity of the kernel. We show how these assumptions are well-suited to the study of kernels depending only on the distance between two points in a metric space, in which case the regularity only depends on the decay of the eigenvalues. This is connected with random geometric graphs, which we study further, giving explicit formulas for the spectrum and its fluctuations.

[1]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[2]  Jiaming Xu,et al.  Rates of Convergence of Spectral Methods for Graphon Estimation , 2017, ICML.

[3]  V. Sós,et al.  Convergent Sequences of Dense Graphs II. Multiway Cuts and Statistical Physics , 2012 .

[4]  Ding-Xuan Zhou,et al.  The covering number in learning theory , 2002, J. Complex..

[5]  Thorsten Gerber,et al.  Handbook Of Mathematical Functions , 2016 .

[6]  U. Feige,et al.  Spectral Graph Theory , 2015 .

[7]  S. Boucheron,et al.  Concentration inequalities : a non asymptotic theory of independence , 2013 .

[8]  Serge Nicaise.  Jacobi Polynomials, Weighted Sobolev Spaces and Approximation Results of Some Singularities , 2000 .

[9]  Yuan Xu,et al.  Approximation Theory and Harmonic Analysis on Spheres and Balls , 2013 .

[10]  A. Tsybakov,et al.  Oracle inequalities for network models and sparse graphon estimation , 2015, 1507.04118.

[11]  Avraham Adler,et al.  Lambert-W Function , 2015 .

[12]  I. Krasikov.  New bounds on the Hermite polynomials , 2004, math/0401310.

[13]  G. Fasshauer.  Positive definite kernels: past, present and future , 2011 .

[14]  E FasshauerG.  Positive definite kernels: past, present and future , 2011 .

[15]  Sébastien Bubeck,et al.  Testing for high‐dimensional geometry in random graphs , 2014, Random Struct. Algorithms.

[16]  Ioannis Chatzigeorgiou,et al.  Bounds on the Lambert Function and Their Application to the Outage Analysis of User Cooperation , 2013, IEEE Communications Letters.

[17]  J. Dall,et al.  Random geometric graphs. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[18]  Nello Cristianini,et al.  On the eigenspectrum of the gram matrix and the generalization error of kernel-PCA , 2005, IEEE Transactions on Information Theory.

[19]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[20]  Y. D. Castro,et al.  Adaptive estimation of nonparametric geometric graphs , 2017, Mathematical Statistics and Learning.

[21]  Gaston H. Gonnet,et al.  On the LambertW function , 1996, Adv. Comput. Math..

[22]  M. Bálek,et al.  Large Networks and Graph Limits , 2022 .

[23]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[24]  Mikio L. Braun,et al.  Accurate Error Bounds for the Eigenvalues of the Kernel Matrix , 2006, J. Mach. Learn. Res..