Training sparse SVM on the core sets of fitting-planes

Controlling the sparsity of a classifier is a key to train SVM efficiently on very large scale problems. This paper explores building SVM classifier on the fitting-plane of each class of data, which captures the distributing trend of the corresponding class of data. The newly developed plane-fitting model can be solved by core set methods, and the SVM is trained only on the core sets which are small subsets of the original data. The computing complexity of the proposed algorithm is up bounded by O(1/@e). Experimental results show that the new algorithm scales better than SVMperf and CVM/BVM, while their predicting accuracies are almost comparable.

[1]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[2]  Su-Yun Huang,et al.  Reduced Support Vector Machines: A Statistical Theory , 2007, IEEE Transactions on Neural Networks.

[3]  Tom Downs,et al.  Exact Simplification of Support Vector Solutions , 2002, J. Mach. Learn. Res..

[4]  Jacek M. Zurada,et al.  Generalized Core Vector Machines , 2006, IEEE Transactions on Neural Networks.

[5]  S. Sathiya Keerthi,et al.  Building Support Vector Machines with Reduced Classifier Complexity , 2006, J. Mach. Learn. Res..

[6]  Christopher J. C. Burges,et al.  Simplified Support Vector Decision Rules , 1996, ICML.

[7]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[8]  Bernhard Schölkopf,et al.  Sparse Greedy Matrix Approximation for Machine Learning , 2000, International Conference on Machine Learning.

[9]  Michael I. Jordan,et al.  Predictive low-rank decomposition for kernel methods , 2005, ICML.

[10]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[11]  Xin Yao,et al.  Sparse Approximation Through Boosting for Learning Large Scale Kernel Machines , 2010, IEEE Transactions on Neural Networks.

[12]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[13]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Xinyu Guo,et al.  Pruning Support Vector Machines Without Altering Performances , 2008, IEEE Transactions on Neural Networks.

[15]  Thorsten Joachims,et al.  Sparse kernel SVMs via cutting-plane training , 2009, Machine Learning.

[16]  Licheng Jiao,et al.  Fast Sparse Approximation for Least Squares Support Vector Machine , 2007, IEEE Transactions on Neural Networks.

[17]  Bernhard Schölkopf,et al.  Building Sparse Large Margin Classifiers , 2005, ICML.

[18]  Chih-Jen Lin,et al.  Working Set Selection Using Second Order Information for Training Support Vector Machines , 2005, J. Mach. Learn. Res..

[19]  Xinjun Peng,et al.  Building sparse twin support vector machine classifiers in primal space , 2011, Inf. Sci..

[20]  Kenneth L. Clarkson,et al.  Optimal core-sets for balls , 2008, Comput. Geom..

[21]  Chih-Jen Lin,et al.  A study on reduced support vector machines , 2003, IEEE Trans. Neural Networks.