Adaptive simplification of solution for support vector machine

SVM has been receiving increasing interest in areas ranging from its original application in pattern recognition to other applications such as regression estimation due to its remarkable generalization performance. Unfortunately, SVM is currently considerably slower in test phase caused by number of the support vectors, which has been a serious limitation for some applications. To overcome this problem, we proposed an adaptive algorithm named feature vectors selection (FVS) to select the feature vectors from the support vector solutions, which is based on the vector correlation principle and greedy algorithm. Through the adaptive algorithm, the sparsity of solution is improved and the time cost in testing is reduced. To select the number of the feature vectors adaptively by the requirements, the generalization and complexity trade-off can be directly controlled. The computer simulations on regression estimation and pattern recognition show that FVS is a promising algorithm to simplify the solution for support vector machine.

[1]  Patrick Haffner,et al.  Support vector machines for histogram-based image classification , 1999, IEEE Trans. Neural Networks.

[2]  John Shawe-Taylor,et al.  Generalisation Error Bounds for Sparse Linear Classifiers , 2000, COLT.

[3]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[4]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[5]  Christopher J. C. Burges,et al.  Simplified Support Vector Decision Rules , 1996, ICML.

[6]  Francis Eng Hock Tay,et al.  Support vector machine with adaptive parameters in financial time series forecasting , 2003, IEEE Trans. Neural Networks.

[7]  K. Lang,et al.  Learning to tell two spirals apart , 1988 .

[8]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[9]  Tom Downs,et al.  Exact Simplification of Support Vector Solutions , 2002, J. Mach. Learn. Res..

[10]  G. Baudat,et al.  Feature vector selection and projection using kernels , 2003, Neurocomputing.

[11]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[12]  Christopher J. C. Burges,et al.  Geometry and invariance in kernel based methods , 1999 .

[13]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[14]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[15]  Hang Joon Kim,et al.  Support Vector Machines for Texture Classification , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Michael S. Schmidt,et al.  Identifying Speakers With Support Vector Networks , 1996 .

[17]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[18]  Marko Grobelnik,et al.  Feature Selection Using Linear Support Vector Machines , 2002 .