Improving the sample complexity using global data

We study the sample complexity of proper and improper learning problems with respect to different q-loss functions. We improve the known estimates for classes which have relatively small covering numbers in empirical L/sub 2/ spaces (e.g. log-covering numbers which are polynomial with exponent p<2). We present several examples of relevant classes which have a "small" fat-shattering dimension, and hence fit our setup, the most important of which are kernel machines.

[1]  O. Hanner.  On the uniform convexity ofLp andlp , 1956 .

[2]  B. Beauzamy.  Introduction to Banach spaces and their geometry , 1985 .

[3]  R. Dudley.  Universal Donsker Classes and Metric Entropy , 1987 .

[4]  M. Talagrand.  Sharper Bounds for Gaussian and Empirical Processes , 1994 .

[5]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[6]  Wee Sun Lee,et al.  Agnostic learning and single hidden layer neural networks , 1996 .

[7]  Leonid Gurvits,et al.  A Note on a Scale-Sensitive Dimension of Linear Bounded Functionals in Banach Spaces , 1997, ALT.

[8]  B. Carl.  Metric Entropy of Convex Hulls in Hilbert Spaces , 1997 .

[9]  S. Saitoh.  Integral Transforms, Reproducing Kernels and Their Applications , 1997 .

[10]  P. Gänssler.  Weak Convergence and Empirical Processes - A. W. van der Vaart; J. A. Wellner. , 1997 .

[11]  Peter L. Bartlett,et al.  The Importance of Convexity in Learning with Squared Loss , 1998, IEEE Trans. Inf. Theory.

[12]  B. Carl,et al.  Metric Entropy of Convex Hulls in Banach Spaces , 1999 .

[13]  Peter L. Bartlett,et al.  Neural Network Learning - Theoretical Foundations , 1999 .

[14]  P. Massart,et al.  About the constants in Talagrand's concentration inequalities for empirical processes , 2000 .

[15]  Shahar Mendelson,et al.  On the Size of Convex Hulls of Small Sets , 2002, J. Mach. Learn. Res..

[16]  Felipe Cucker,et al.  On the mathematical foundations of learning , 2001 .

[17]  Bernhard Schölkopf,et al.  Generalization Performance of Regularization Networks and Support Vector Machines via Entropy Numbers of Compact Operators , 1998 .

[18]  Shahar Mendelson,et al.  Rademacher averages and phase transitions in Glivenko-Cantelli classes , 2002, IEEE Trans. Inf. Theory.

[19]  Shahar Mendelson,et al.  Geometric Parameters of Kernel Machines , 2002, COLT.

[20]  Shahar Mendelson,et al.  Learnability in Hilbert Spaces with Reproducing Kernels , 2002, J. Complex..