Sparseness of Support Vector Machines

Support vector machines (SVMs) construct decision functions that are linear combinations of kernel evaluations on the training set. The samples with non-vanishing coefficients are called support vectors. In this work we establish lower (asymptotical) bounds on the number of support vectors. On our way we prove several results which are of great importance for the understanding of SVMs. In particular, we describe to which "limit" SVM decision functions tend, discuss the corresponding notion of convergence and provide some results on the stability of SVMs using subdifferential calculus in the associated reproducing kernel Hilbert space.

[1]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[2]  G. Wahba,et al.  Some results on Tchebycheffian spline functions , 1971 .

[3]  J. Diestel,et al.  On vector measures , 1974 .

[4]  R. Rockafellar,et al.  Integral functionals, normal integrands and measurable selections , 1976 .

[5]  C. Berg,et al.  Harmonic Analysis on Semigroups: Theory of Positive Definite and Related Functions , 1984 .

[6]  R. Phelps.  Convex Functions, Monotone Operators and Differentiability , 1989 .

[7]  D. Cox,et al.  Asymptotic Analysis of Penalized Likelihood and Related Estimators , 1990 .

[8]  V. Yurinsky.  Sums and Gaussian Vectors , 1995 .

[9]  G. Romano.  New results in subdifferential calculus with applications to convex optimization , 1995 .

[10]  齋藤 三郎.  Integral transforms, reproducing kernels and their applications , 1997 .

[11]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[12]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[13]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[14]  Klaus Ritter,et al.  Average-case analysis of numerical problems , 2000, Lecture notes in mathematics.

[15]  Tong Zhang,et al.  Convergence of Large Margin Separable Linear Classification , 2000, NIPS.

[16]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[17]  Tom Downs,et al.  Exact Simplification of Support Vector Solutions , 2002, J. Mach. Learn. Res..

[18]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[19]  Dudley,et al.  Real Analysis and Probability: Measurability: Borel Isomorphism and Analytic Sets , 2002 .

[20]  Ingo Steinwart,et al.  Support Vector Machines are Universally Consistent , 2002, J. Complex..

[21]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[22]  Ingo Steinwart,et al.  Sparseness of Support Vector Machines---Some Asymptotically Sharp Bounds , 2003, NIPS.

[23]  Tong Zhang.  Statistical behavior and consistency of classification methods based on convex risk minimization , 2003 .

[24]  Ingo Steinwart,et al.  On the Optimal Parameter Choice for v-Support Vector Machines , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Yi Lin,et al.  Support Vector Machines and the Bayes Rule in Classification , 2002, Data Mining and Knowledge Discovery.

[26]  R. Shah,et al.  Least Squares Support Vector Machines , 2022 .