A Multi-objective Genetic Algorithm for Model Selection for Support Vector Machines

Selecting the proper Kernel function in SVMs and the specific parameters for that kernel is an important step in achieving a high performance learning machine. The objective of this research is to optimize SVMs parameters using different kernel functions. We cast this problem as a multi-objective optimization problem, where the classification accuracy, the number of support vectors and the margin define our objective functions. So, we introduce a method based on multi-objective evolutionary algorithm NSGA-II to solve this problem. We also introduce a multi-criteria selection operator for our NSGA-II. The proposed method is applied on some benchmark datasets. The experimental obtained results show the efficiency of the proposed method.

[1]  Chih-Jen Lin,et al.  Radius Margin Bounds for Support Vector Machines with the RBF Kernel , 2002, Neural Computation.

[2]  Kalyanmoy Deb,et al.  Omni-optimizer: A generic evolutionary algorithm for single and multi-objective optimization , 2008, Eur. J. Oper. Res..

[3]  Kalyanmoy Deb,et al.  Self-Adaptive Genetic Algorithms with Simulated Binary Crossover , 2001, Evolutionary Computation.

[4]  Ioannis Pitas,et al.  Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines , 2007, IEEE Transactions on Image Processing.

[5]  Christian Igel,et al.  Evolutionary tuning of multiple SVM parameters , 2005, ESANN.

[6]  Chih-Jen Lin,et al.  Asymptotic Behaviors of Support Vector Machines with Gaussian Kernel , 2003, Neural Computation.

[7]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[8]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[9]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[10]  Huanhuan Chen,et al.  Evolving Least Squares Support Vector Machines for Stock Market Trend Mining , 2009, IEEE Trans. Evol. Comput..

[11]  Mingtian Zhou,et al.  Feature selection and parameter optimization for support vector machines: A new approach based on genetic algorithm with feature chromosomes , 2011, Expert Syst. Appl..

[12]  Liu Fang,et al.  Choosing multiple parameters for SVM based on genetic algorithm , 2002, 6th International Conference on Signal Processing, 2002..

[13]  Cheng-Lung Huang,et al.  A GA-based feature selection and parameters optimizationfor support vector machines , 2006, Expert Syst. Appl..

[14]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[15]  J. Rajapakse,et al.  Gene Classification Using Codon Usage and Support Vector Machines , 2009, IEEE/ACM Transactions on Computational Biology & Bioinformatics.

[16]  Yao-Nan Wang,et al.  A method to choose kernel function and its parameters for support vector machines , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[17]  Shih-Wei Lin,et al.  Particle swarm optimization for parameter determination and feature selection of support vector machines , 2008, Expert Syst. Appl..

[18]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[19]  Shuang Liu,et al.  A new weighted support vector machine with GA-based parameter selection , 2005, 2005 International Conference on Machine Learning and Cybernetics.