Exact SVM training by Wolfe's minimum norm point algorithm

This paper applies Wolfe's algorithm for finding the minimum norm point in a polytope to training of standard SVM with hinge loss. The resulting algorithm is guaranteed to obtain an exact optimal solution within a finite number of iterations. Experiments illustrate that our algorithm runs faster than existing algorithms such as LIBSVM for the same model. In comparison with LIBLINEAR, which adopts a variant of SVMs, our approach works better when the feature size is modest; the feature size is sufficiently smaller than the sample size.

[1]  Manuel Blum,et al.  Time Bounds for Selection , 1973, J. Comput. Syst. Sci..

[2]  Jue Wang,et al.  A generalized S-K algorithm for learning v-SVM classifiers , 2004, Pattern Recognit. Lett..

[3]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[4]  V. N. Malozemov,et al.  Finding the Point of a Polyhedron Closest to the Origin , 1974 .

[5]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[6]  Russ Bubley,et al.  Randomized algorithms , 1995, CSUR.

[7]  José R. Dorronsoro,et al.  Clipping algorithms for solving the nearest point problem over reduced convex hulls , 2011, Pattern Recognit..

[8]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2011, Math. Program..

[9]  Václav Hlavác,et al.  An iterative algorithm learning the maximal margin classifier , 2003, Pattern Recognit..

[10]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[11]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[12]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[13]  David H. Mathews,et al.  Detection of non-coding RNAs on the basis of predicted secondary structure formation free energy change , 2006, BMC Bioinformatics.

[14]  Nesa L'abbe Wu,et al.  Linear programming and extensions , 1981 .

[15]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[16]  Sergios Theodoridis,et al.  A geometric approach to Support Vector Machine (SVM) classification , 2006, IEEE Transactions on Neural Networks.

[17]  Philip Wolfe,et al.  Finding the nearest point in A polytope , 1976, Math. Program..

[18]  A. Sideris,et al.  A Proximity Algorithm for Support Vector Machine Classification , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[19]  David J. Crisp,et al.  A Geometric Interpretation of ?-SVM Classifiers , 1999, NIPS 2000.

[20]  S. Sathiya Keerthi,et al.  A fast iterative nearest point algorithm for support vector machine classifier design , 2000, IEEE Trans. Neural Networks Learn. Syst..

[21]  E. Gilbert.  An Iterative Procedure for Computing the Minimum of a Quadratic Form on a Convex Set , 1966 .

[22]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[23]  Kristin P. Bennett,et al.  Duality and Geometry in SVM Classifiers , 2000, ICML.