On Properties of Support Vector Machines for Pattern Recognition in Finite Samples

The support vector machine proposed by Vapnik belongs to a class of modern statistical learning methods based on convex risk minimization. Other special cases are AdaBoost, kernel logistic regression and least squares. The support vector machine has the advantage that it usually leads to a reduction of complexity, because only the support vectors and not all observations contribute to the prediction of a new response. This paper addresses robustness properties of the support vector machine for pattern recognition in finite samples. Sensitivity curves in the sense of J. W. Tukey are used to investigate the possible impact of a single data point.

[1]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[2]  Ingo Steinwart,et al.  Consistency of support vector machines and other regularized kernel classifiers , 2005, IEEE Transactions on Information Theory.

[3]  SteinwartIngo,et al.  On Robustness Properties of Convex Risk Minimization Methods for Pattern Recognition , 2004 .

[4]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[5]  P. L. Davies.  Aspects of Robust Linear Regression , 1993 .

[6]  Johan A. K. Suykens,et al.  Weighted least squares support vector machines: robustness and sparse approximation , 2002, Neurocomputing.

[7]  Ingo Steinwart,et al.  Support Vector Machines are Universally Consistent , 2002, J. Complex..

[8]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machines , 2002 .

[9]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[10]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[11]  Andreas Christmann,et al.  Measuring overlap in binary regression , 2001 .

[12]  Ingo Steinwart,et al.  Sparseness of Support Vector Machines , 2003, J. Mach. Learn. Res..

[13]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[14]  Marti A. Hearst.  Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[15]  Regina Y. Liu,et al.  Regression depth. Commentaries. Rejoinder , 1999 .

[16]  Andreas Christmann,et al.  On Robustness Properties of Convex Risk Minimization Methods for Pattern Recognition , 2004, J. Mach. Learn. Res..

[17]  Andreas Christmann,et al.  Robustness against separation and outliers in logistic regression , 2003, Comput. Stat. Data Anal..

[18]  Hans Ulrich Simon,et al.  Robust Trainability of Single Neurons , 1995, J. Comput. Syst. Sci..

[19]  Tong Zhang.  Statistical behavior and consistency of classification methods based on convex risk minimization , 2003 .

[20]  Thorsten Joachims,et al.  Comparison between various regression depth methods and the support vector machine to approximate the minimum number of missclassifications , 2002, Comput. Stat..

[21]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[22]  Andreas Christmann,et al.  Measuring overlap in logistic regression , 1999 .

[23]  D. Ruppert.  Robust Statistics: The Approach Based on Influence Functions , 1987 .