On the rate of convergence for multi-category classification based on convex losses

The multi-category classification algorithms play an important role in both theory and practice of machine learning. In this paper, we consider an approach to the multi-category classification based on minimizing a convex surrogate of the nonstandard misclassification loss. We bound the excess misclassification error by the excess convex risk. We construct an adaptive procedure to search the classifier and furthermore obtain its convergence rate to the Bayes rule.

[1]  Tong Zhang,et al.  Statistical Analysis of Some Multi-Category Large Margin Classification Methods , 2004, J. Mach. Learn. Res..

[2]  Jason Weston,et al.  Multi-Class Support Vector Machines , 1998 .

[3]  G. Wahba,et al.  Multicategory Support Vector Machines , Theory , and Application to the Classification of Microarray Data and Satellite Radiance Data , 2004 .

[4]  Tomaso A. Poggio,et al.  Regularization Networks and Support Vector Machines , 2000, Adv. Comput. Math..

[5]  Michael I. Jordan,et al.  Convexity, Classification, and Risk Bounds , 2006 .

[6]  Yiming Ying,et al.  Support Vector Machine Soft Margin Classifiers: Error Analysis , 2004, J. Mach. Learn. Res..

[7]  Tong Zhang.  Statistical behavior and consistency of classification methods based on convex risk minimization , 2003 .

[8]  Gilles Blanchard,et al.  On the Rate of Convergence of Regularized Boosting Classifiers , 2003, J. Mach. Learn. Res..

[9]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[10]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[11]  Koby Crammer,et al.  On the Learnability and Design of Output Codes for Multiclass Problems , 2002, Machine Learning.

[12]  Peter L. Bartlett,et al.  Rademacher and Gaussian Complexities: Risk Bounds and Structural Results , 2003, J. Mach. Learn. Res..

[13]  Dao-Hong Xiang,et al.  The consistency of multicategory support vector machines , 2006, Adv. Comput. Math..

[14]  Shie Mannor,et al.  Greedy Algorithms for Classification -- Consistency, Convergence Rates, and Adaptivity , 2003, J. Mach. Learn. Res..

[15]  Ron Meir,et al.  Data-Dependent Bounds for Multi-category Classification Based on Convex Losses , 2003, COLT.

[16]  G. Lugosi,et al.  On the Bayes-risk consistency of regularized boosting methods , 2003 .

[17]  Ralf Herbrich,et al.  Learning Kernel Classifiers: Theory and Algorithms , 2001 .

[18]  Ingo Steinwart,et al.  Consistency of support vector machines and other regularized kernel classifiers , 2005, IEEE Transactions on Information Theory.

[19]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[20]  Tao Sun,et al.  Consistency of Multiclass Empirical Risk Minimization Methods Based on Convex Loss , 2006, J. Mach. Learn. Res..

[21]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[22]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .