An intelligent model selection scheme based on particle swarm optimization

To improve the learning efficiency of support vector machine, an intelligent model selection scheme based on particle swarm optimization (PSO) was presented to optimize the hyper-parameters. By taking the model selection problem as a multi-object optimization problem, one can obtain a solution set known as Pareto front; each one model in this set is non-dominated. PSO was used to solve the above muti-objective optimization problem and then the model set was obtained. The scheme was tested on several datasets, the results show that Pareto front can be obtained in one trial and the effect of every single parameter can be displayed more directly.

[1]  Yves Lecourtier,et al.  Multi-Objective Optimization for SVM Model Selection , 2007, Ninth International Conference on Document Analysis and Recognition (ICDAR 2007).

[2]  Christian Igel,et al.  Uncertainty Handling in Model Selection for Support Vector Machines , 2008, PPSN.

[3]  M. N. Vrahatis,et al.  Particle swarm optimization method in multiobjective problems , 2002, SAC '02.

[4]  Ricardo H. C. Takahashi,et al.  Improving generalization of MLPs with multi-objective optimization , 2000, Neurocomputing.

[5]  Yunqian Ma,et al.  Practical selection of SVM parameters and noise estimation for SVM regression , 2004, Neural Networks.

[6]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[7]  Carlos A. Coello Coello,et al.  Handling multiple objectives with particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[8]  Gavin C. Cawley,et al.  Leave-One-Out Cross-Validation Based Model Selection Criteria for Weighted LS-SVMs , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[9]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[10]  Russell C. Eberhart,et al.  Multiobjective optimization using dynamic neighborhood particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[11]  Tobias Glasmachers,et al.  Degeneracy in model selection for SVMs with radial Gaussian kernel , 2006, ESANN.

[12]  Christophe Charrier,et al.  International Journal of Neural Systems Special Issue on Issue's Topic C World Scientific Publishing Company Tabu Search Model Selection for Svm , 2022 .

[13]  Ching Y. Suen,et al.  Automatic model selection for the optimization of SVM kernels , 2005, Pattern Recognit..

[14]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[15]  Tomohiro Ando,et al.  Bayesian Model Averaging and Bayesian Predictive Information Criterion for Model Selection , 2008 .

[16]  Hugo Jair Escalante,et al.  Particle Swarm Model Selection , 2009, J. Mach. Learn. Res..

[17]  Christian Igel,et al.  Evolutionary tuning of multiple SVM parameters , 2005, ESANN.

[18]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[19]  Olivier Chapelle,et al.  Model Selection for Support Vector Machines , 1999, NIPS.