论文信息 - Rademacher Complexity Margin Bounds for Learning with a Large Number of Classes

Rademacher Complexity Margin Bounds for Learning with a Large Number of Classes

This paper presents improved Rademacher complexity margin bounds that scale linearly with the number of classes as opposed to the quadratic dependence of existing Rademacher complexity margin-based learning guarantees. We further use this result to prove a novel generalization bound for multi-class classifier ensembles that depends only on the Rademacher complexity of the hypothesis classes to which the classifiers in the ensemble belong.

M. Mohri | Umar Syed | Vitaly Kuznetsov

[1] Yoav Freund,et al. A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[2] Yoav Freund,et al. Boosting the margin: A new explanation for the effectiveness of voting methods , 1997, ICML.

[3] Dale Schuurmans,et al. Boosting in the Limit: Maximizing the Margin of Learned Ensembles , 1998, AAAI/IAAI.

[4] Robert E. Schapire,et al. Theoretical Views of Boosting and Applications , 1999, ALT.

[5] J. Friedman. Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[6] Gunnar Rätsch,et al. On the Convergence of Leveraging , 2001, NIPS.

[7] V. Koltchinskii,et al. Empirical margin distributions and bounding the generalization error of combined classifiers , 2002, math/0405343.

[8] Gunnar Rätsch,et al. Soft Margins for AdaBoost , 2001, Machine Learning.

[9] Thomas G. Dietterich. An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[10] Yoram Singer,et al. Boosting with structural sparsity , 2009, ICML '09.

[11] Ameet Talwalkar,et al. Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[12] Mehryar Mohri,et al. Deep Boosting , 2014, ICML.

[13] Mehryar Mohri,et al. Multi-Class Deep Boosting , 2014, NIPS.