Accurate Probabilistic Error Bound for Eigenvalues of Kernel Matrix

The eigenvalues of the kernel matrix play an important role in a number of kernel methods. It is well known that these eigenvalues converge as the number of samples tends to infinity. We derive a probabilistic finite sample size bound on the approximation error of an individual eigenvalue, which has the important property that the bound scales with the dominate eigenvalue under consideration, reflecting the accurate behavior of the approximation error as predicted by asymptotic results and observed in numerical simulations. Under practical conditions, the bound presented here forms a significant improvement over existing non-scaling bound. Applications of this theoretical finding in kernel matrix selection and kernel target alignment are also presented.