An optimization method for selecting parameters in support vector machines

It has been shown that the cost parameters and kernel parameters are critical in the performance of support vector machines (SVMs). A standard parameter selection method compares parameters among a discrete set of values, called the candidate set, and picks the one which has the best classification accuracy. As a result, the choice of parameters strongly depends on the pre-defined candidate set. In this paper, we formulate the selection of the cost parameter and kernel parameter as a two-level optimization problem, in which the values of parameters vary continuously and thus optimization techniques can be applied to select ideal parameters. Due to the non-smoothness of the objective function in our model, a genetic algorithm has been presented. Numerical results show that the two-level approach can significantly improve the performance of SVM classifier in terms of classification accuracy.

[1]  Colin R. Reeves,et al.  Genetic Algorithms: Principles and Perspectives: A Guide to Ga Theory , 2002 .

[2]  Yuh-Jye Lee,et al.  SSVM: A Smooth Support Vector Machine for Classification , 2001, Comput. Optim. Appl..

[3]  Raúl Hector Gallard,et al.  Genetic algorithms + Data structure = Evolution programs , Zbigniew Michalewicz , 1999 .

[4]  Robert Tibshirani,et al.  The Entire Regularization Path for the Support Vector Machine , 2004, J. Mach. Learn. Res..

[5]  Zbigniew Michalewicz,et al.  Handling Constraints in Genetic Algorithms , 1991, ICGA.

[6]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[7]  Chih-Jen Lin,et al.  A Simple Decomposition Method for Support Vector Machines , 2002, Machine Learning.

[8]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[9]  G. E. Liepins,et al.  A Genetic Algorithm Approach to Multiple-Fault Diagnosis , 1991 .

[10]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[11]  Kalyanmoy Deb,et al.  Don't Worry, Be Messy , 1991, ICGA.

[12]  K. Schittkowski.  Optimal parameter selection in support vector machines , 2005 .

[13]  Thorsten Joachims,et al.  Making large-scale support vector machine learning practical , 1999 .

[14]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[15]  Steven M. LaValle,et al.  On the Relationship between Classical Grid Search and Probabilistic Roadmaps , 2004, Int. J. Robotics Res..