A geometric method for model selection in support vector machine

Support vector machine (SVM) has become one of the most popular methods in machine learning during the last years. The parameters' selection in SVM is an important step in achieving a high performance learning machine. Some methods are proposed by minimizing an estimate of generalization error based on bound of leave-one-out (LOO) bound, empirical error, etc. These methods have to optimize many quadratic programming problems and compute an inversion of the Gram-Schmidt matrix, which cause to be time-consuming in large-scale problems. This paper introduces a fast incremental method to optimize the kernel parameters in SVM by combining a geometric algorithm on SVM and an approximation of the gradient of the empirical error. This method shows an online way to update the kernel parameters and work set in incremental learning, which reduces the resources required both CPU time and storage space. The numerical tests on some benchmarks confirm our method.

[1]  B. Dong,et al.  Applying support vector machines to predict building energy consumption in tropical region , 2005 .

[2]  Chih-Jen Lin,et al.  Radius Margin Bounds for Support Vector Machines with the RBF Kernel , 2002, Neural Computation.

[3]  K. Schittkowski.  Optimal parameter selection in support vector machines , 2005 .

[4]  Federico Girosi,et al.  Training support vector machines: an application to face detection , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[6]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[7]  Mohamed Cheriet,et al.  Optimizing resources in model selection for support vector machines , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[8]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[9]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[10]  Ching Y. Suen,et al.  Automatic model selection for the optimization of SVM kernels , 2005, Pattern Recognit..

[11]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[12]  Sergios Theodoridis,et al.  A geometric approach to Support Vector Machine (SVM) classification , 2006, IEEE Transactions on Neural Networks.