Learnability in Hilbert Spaces with Reproducing Kernels

We explore the question of the learnability of classes of functions contained in a Hilbert space which has a reproducing kernel. We show that if the evaluation functionals are uniformly bounded and if the class is norm bounded then it is learnable. We formulate a learning procedure related to the well known support vector machine (SVM), which requires solving a system of linear equations, rather than the quadratic programming needed for the SVM. As a part of our discussion, we estimate the fat-shattering dimension of the unit ball of the dual of a Banach space when considered as a set of functions on the unit ball of the space itself. Our estimate is based on a geometric property of the Banach space called the type.

[1]  R. Dudley,et al.  Uniform Central Limit Theorems: Notation Index , 2014 .

[2]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[3]  R. Dudley,et al.  Uniform and universal Glivenko-Cantelli classes , 1991 .

[4]  Shahar Mendelson,et al.  On the Size of Convex Hulls of Small Sets , 2002, J. Mach. Learn. Res..

[5]  Shahar Mendelson.  Geometric Methods in the Analysis of Glivenko-Cantelli Classes , 2001, COLT/EuroCOLT.

[6]  E. Berger.  UNIFORM CENTRAL LIMIT THEOREMS (Cambridge Studies in Advanced Mathematics 63) By R. M. D UDLEY : 436pp., £55.00, ISBN 0-521-46102-2 (Cambridge University Press, 1999). , 2001 .

[7]  Philip M. Long,et al.  More theorems about scale-sensitive dimensions and learning , 1995, COLT '95.

[8]  S. Saitoh.  Integral Transforms, Reproducing Kernels and Their Applications , 1997 .

[9]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[10]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[11]  N. Tomczak-Jaegermann.  Banach-Mazur distances and finite-dimensional operator ideals , 1989 .

[12]  Joram Lindenstrauss,et al.  Classical Banach spaces , 1973 .

[13]  R. Dudley.  A course on empirical processes , 1984 .

[14]  Noga Alon,et al.  Scale-sensitive dimensions, uniform convergence, and learnability , 1997, JACM.

[15]  R. Dudley.  Uniform Central Limit Theorems: Preface , 1999 .

[16]  D. Gilbarg,et al.  Elliptic Partial Differential Equa-tions of Second Order , 1977 .

[17]  G. Pisier.  Probabilistic methods in the geometry of Banach spaces , 1986 .

[18]  Robert E. Schapire,et al.  Efficient distribution-free learning of probabilistic concepts , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[19]  Leonid Gurvits.  A note on a scale-sensitive dimension of linear bounded functionals in Banach spaces , 2001, Theor. Comput. Sci..

[20]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .