An Adaptive Support Vector Machine Learning Algorithm for Large Classification Problem

Based on the incremental and decremental learning strategies, an adaptive support vector machine learning algorithm (ASVM) is presented for large classification problems in this paper. In the proposed algorithm, the incremental and decremental procedures are performed alternatively, and a small scale working set, which can cover most of the information in the training set and overcome the drawback of losing the sparseness in least squares support vector machine (LS-SVM), can be formed adaptively. The classifier can be constructed by using this working set. In general, the number of the elements in the working set is much smaller than that in the training set. Therefore the proposed algorithm can be used not only to train the data sets quickly but also to test them effectively with losing little accuracy. In order to examine the training speed and the generalization performance of the proposed algorithm, we apply both ASVM and LS-SVM to seven UCI datasets and a benchmark problem. Experimental results show that the novel algorithm is very faster than LS-SVM and loses little accuracy in solving large classification problems.

[1]  Yanchun Liang,et al.  An extended Lagrangian support vector machine for classifications , 2004 .

[2]  J. Platt.  Sequential Minimal Optimization : A Fast Algorithm for Training Support Vector Machines , 1998 .

[3]  Federico Girosi,et al.  An improved training algorithm for support vector machines , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[4]  David R. Musicant,et al.  Successive overrelaxation for support vector machines , 1999, IEEE Trans. Neural Networks.

[5]  Katya Scheinberg,et al.  Efficient SVM Training Using Low-Rank Kernel Representations , 2002, J. Mach. Learn. Res..

[6]  Johan A. K. Suykens,et al.  Benchmarking Least Squares Support Vector Machine Classifiers , 2004, Machine Learning.

[7]  S. Sathiya Keerthi,et al.  Convergence of a Generalized SMO Algorithm for SVM Classifier Design , 2002, Machine Learning.

[8]  JianghuaLiu,et al.  Online LS-SVM for function estimation and classification , 2003 .

[9]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[10]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[11]  Sun Jian.  An Improved Sequential Minimization Optimization Algorithm for Support Vector Machine Training , 2002 .

[12]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[13]  Johan A. K. Suykens,et al.  Sparse approximation using least squares support vector machines , 2000, 2000 IEEE International Symposium on Circuits and Systems. Emerging Technologies for the 21st Century. Proceedings (IEEE Cat No.00CH36353).

[14]  Chih-Jen Lin,et al.  On the convergence of the decomposition method for support vector machines , 2001, IEEE Trans. Neural Networks.

[15]  Thorsten Joachims,et al.  Making large-scale support vector machine learning practical , 1999 .

[16]  Q. Henry Wu,et al.  Online training of support vector classifier , 2003, Pattern Recognit..

[17]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[18]  Kok Seng Chua,et al.  Efficient computations for large least square support vector machine classifiers , 2003, Pattern Recognit. Lett..

[19]  S. Sathiya Keerthi,et al.  Improvements to Platt's SMO Algorithm for SVM Classifier Design , 2001, Neural Computation.

[21]  Li Jian-Min,et al.  An Improvement Algorithm to Sequential Minimal Optimization , 2003 .

[22]  David R. Musicant,et al.  Lagrangian Support Vector Machines , 2001, J. Mach. Learn. Res..

[23]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .