Random Subclass Bounds

It has been recently shown that sharp generalization bounds can be obtained when the function class from which the algorithm choo-ses its hypotheses is “small” in the sense that the Rademacher averages of this function class are small [8,9]. Seemingly based on different arguments, generalization bounds were obtained in the compression scheme [7], luckiness [13], and algorithmic luckiness [6] frameworks in which the “size” of the function class is not specified a priori.

[1]  Y. Gat.  A bound concerning the generalization ability of a certain class of learning algorithms , 1999 .

[2]  John Shawe-Taylor,et al.  Structural Risk Minimization Over Data-Dependent Hierarchies , 1998, IEEE Trans. Inf. Theory.

[3]  Shahar Mendelson,et al.  Improving the sample complexity using global data , 2002, IEEE Trans. Inf. Theory.

[4]  P. Bartlett,et al.  Local Rademacher complexities , 2005, math/0508275.

[5]  Manfred K. Warmuth,et al.  Relating Data Compression and Learnability , 2003 .

[6]  Don R. Hush,et al.  Machine Learning with Data Dependent Hypothesis Classes , 2002, J. Mach. Learn. Res..

[7]  Shahar Mendelson,et al.  A Few Notes on Statistical Learning Theory , 2002, Machine Learning Summer School.

[8]  Ralf Herbrich,et al.  Algorithmic Luckiness , 2001, J. Mach. Learn. Res..

[9]  Manfred K. Warmuth,et al.  Sample compression, learnability, and the Vapnik-Chervonenkis dimension , 1995, Machine Learning.

[10]  Shahar Mendelson,et al.  On the Importance of Small Coordinate Projections , 2004, J. Mach. Learn. Res..

[11]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[12]  Gunnar Rätsch,et al.  Advanced Lectures on Machine Learning , 2004, Lecture Notes in Computer Science.

[13]  Shahar Mendelson,et al.  Rademacher averages and phase transitions in Glivenko-Cantelli classes , 2002, IEEE Trans. Inf. Theory.

[14]  John Shawe-Taylor,et al.  Generalisation Error Bounds for Sparse Linear Classifiers , 2000, COLT.

[15]  M. Talagrand.  Majorizing measures: the generic chaining , 1996 .