Robust support vector machines for classification and computational issues

In this paper, we investigate the theoretical and numerical aspects of robust classification using support vector machines (SVMs) by providing second order cone programming and linear programming formulations. SVMs are learning algorithms introduced by Vapnik used either for classification or regression. They show good generalization properties and they are based on statistical learning theory. The resulting learning problems are convex optimization problems suitable for application of primal-dual interior points methods. We investigate the training of a SVM in the case where a bounded perturbation is added to the value of an input x i ∈ℝ n . A robust SVM provides a decision function that is immune to data perturbations. We consider both cases where our training data are either linearly separable or non linearly separable respectively and provide computational results for real data sets.

[1]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[2]  Paul Horton,et al.  A Probabilistic Classification System for Predicting the Cellular Localization Sites of Proteins , 1996, ISMB.

[3]  Bernhard Schölkopf,et al.  Support vector learning , 1997 .

[4]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[5]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[6]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[7]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[8]  J. C. BurgesChristopher.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998 .

[9]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[10]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[11]  Stephen P. Boyd,et al.  Applications of second-order cone programming , 1998 .

[12]  Nello Cristianini,et al.  Large Margin DAGs for Multiclass Classification , 1999, NIPS.

[13]  Arkadi Nemirovski,et al.  Robust solutions of uncertain linear programs , 1999, Oper. Res. Lett..

[14]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[15]  Kristin P. Bennett,et al.  Support vector machines: hype or hallelujah? , 2000, SKDD.

[16]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[17]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[18]  T. Trafalis,et al.  Robust optimization in support vector machine training with bounded errors , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[19]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.