An efficient model selection for SVM in realworld datasets using BGA and RGA

Supp ort vector machine (SVM) has become one of the most popular machine-learning methods during the last years. The design of an efficient model and the proper adjustment of the SVMs parameters are integral to reducing the testing time and enhancing performance. In this paper, a new bipartite objective function consisted of the sparseness property and generalization performance is proposed. Since the proposed objective function is based on selecting fewer num- bers of the support vectors, the model complexity is reduced while the performance accuracy remains at an acceptable level. Due to the model complexity reduction, the testing time is decreased and the ability of SVM in practical applications is increased Moreover, to prove the performance of the proposed objective function, a comparative study was carried out on the proposed objective function and the conventional objective function, which is only based on the generalization perfor- mance, using the Binary Genetic Algorithm (BGA) and Real-valued vectors GA (RGA). The effectiveness of the proposed cost function is demonstrated based on the results of the comparative study on four real-world datasets of UCI database.

[1]  Shih-Wei Lin,et al.  Particle swarm optimization for parameter determination and feature selection of support vector machines , 2008, Expert Syst. Appl..

[2]  Alexander J. Smola,et al.  Advances in Large Margin Classifiers , 2000 .

[3]  Jason Weston,et al.  Breaking SVM Complexity with Cross-Training , 2004, NIPS.

[4]  Peifeng Niu,et al.  LS-SVM based on Chaotic Particle Swarm Optimization with simulated annealing and application , 2011, 2011 2nd International Conference on Intelligent Control and Information Processing.

[5]  Yatong Zhou,et al.  Analysis of the Distance Between Two Classes for Tuning SVM Hyperparameters , 2010, IEEE Transactions on Neural Networks.

[6]  Ashraf Saad,et al.  Metaheuristic techniques for Support Vector Machine model selection , 2010, 2010 10th International Conference on Hybrid Intelligent Systems.

[7]  Su-Yun Huang,et al.  Incremental Reduced Support Vector Machines , 2001 .

[8]  Andrei Lihu,et al.  Real-valued genetic algorithms with disagreements , 2012, Memetic Comput..

[9]  Bo Meng,et al.  Parameter Selection Algorithm for Support Vector Machine , 2011 .

[10]  Christian Igel,et al.  Gradient-Based Adaptation of General Gaussian Kernels , 2005, Neural Computation.

[11]  Tianyou Chai,et al.  An adaptive chaotic PSO for parameter optimization and feature extraction of LS-SVM based modelling , 2011, Proceedings of the 2011 American Control Conference.

[12]  X. C. Guo,et al.  A novel LS-SVMs hyper-parameter selection based on particle swarm optimization , 2008, Neurocomputing.

[13]  Zhiyong Luo,et al.  SVM parameters tuning with quantum particles swarm optimization , 2008, 2008 IEEE Conference on Cybernetics and Intelligent Systems.

[14]  V. Vapnik,et al.  Bounds on Error Expectation for Support Vector Machines , 2000, Neural Computation.

[15]  Sheng Ding,et al.  Evolutionary Computing Optimization for Parameter Determination and Feature Selection of Support Vector Machines , 2009, 2009 International Conference on Computational Intelligence and Software Engineering.

[16]  Qing Li,et al.  Adaptive simplification of solution for support vector machine , 2007, Pattern Recognit..

[17]  Sayan Mukherjee,et al.  Choosing Multiple Parameters for Support Vector Machines , 2002, Machine Learning.

[18]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[19]  Cheng-Lung Huang,et al.  A GA-based feature selection and parameters optimizationfor support vector machines , 2006, Expert Syst. Appl..

[20]  S. Sathiya Keerthi,et al.  Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms , 2002, IEEE Trans. Neural Networks.

[21]  Jiawei Han,et al.  Classifying large data sets using SVMs with hierarchical clusters , 2003, KDD '03.

[22]  Adeike A. Adewuya.  New methods in genetic search with real-valued chromosomes , 1996 .

[23]  Bernhard Schölkopf,et al.  Feature selection for support vector machines by means of genetic algorithm , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[24]  Jih Pin Yeh,et al.  A hybrid optimization strategy for simplifying the solutions of support vector machines , 2010, Pattern Recognit. Lett..

[25]  Yi Luo,et al.  Parameters Selection of Support Vector Machine Using an Improved PSO Algorithm , 2010, 2010 Second International Conference on Intelligent Human-Machine Systems and Cybernetics.

[26]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[27]  Chih-Hung Wu,et al.  A real-valued genetic algorithm to optimize the parameters of support vector machine for predicting bankruptcy , 2007, Expert Syst. Appl..

[28]  Jason Weston,et al.  Fast Kernel Classifiers with Online and Active Learning , 2005, J. Mach. Learn. Res..

[29]  Peter Williams,et al.  A Geometrical Method to Improve Performance of the Support Vector Machine , 2007, IEEE Transactions on Neural Networks.

[30]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[31]  Bo Meng,et al.  PSO Algorithm for Support Vector Machine , 2010, 2010 Third International Symposium on Electronic Commerce and Security.

[32]  Chih-Jen Lin,et al.  Asymptotic Behaviors of Support Vector Machines with Gaussian Kernel , 2003, Neural Computation.

[33]  Yuh-Jye Lee,et al.  Clustering Model Selection for Reduced Support Vector Machines , 2004, IDEAL.

[34]  Manfred Opper,et al.  Advances in large margin classifiers , 2000 .

[35]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[36]  David Haussler,et al.  Probabilistic kernel regression models , 1999, AISTATS.

[37]  Yifei Wang,et al.  A geometric method for model selection in support vector machine , 2009, Expert Syst. Appl..

[38]  Annabella Astorino,et al.  Scaling Up Support Vector Machines Using Nearest Neighbor Condensation , 2010, IEEE Transactions on Neural Networks.

[39]  Tom Downs,et al.  Exact Simplification of Support Vector Solutions , 2002, J. Mach. Learn. Res..

[40]  Hsuan-Tien Lin.  A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods , 2005 .