Optimizing Support Vector Machine parameters using continuous Ant Colony Optimization

Support Vector Machines are considered to be excellent patterns classification techniques. The process of classifying a pattern with high classification accuracy counts mainly on tuning Support Vector Machine parameters which are the generalization error parameter and the kernel function parameter. Tuning these parameters is a complex process and may be done experimentally through time consuming human experience. To overcome this difficulty, an approach such as Ant Colony Optimization can tune Support Vector Machine parameters. Ant Colony Optimization originally deals with discrete optimization problems. Hence, in applying Ant Colony Optimization for optimizing Support Vector Machine parameters, which are continuous parameters, there is a need to discretize the continuous value into a discrete value. This discretization process results in loss of some information and, hence, affects the classification accuracy and seek time. This study proposes an algorithm to optimize Support Vector Machine parameters using continuous Ant Colony Optimization without the need to discretize continuous values for Support Vector Machine parameters. Seven datasets from UCI were used to evaluate the performance of the proposed hybrid algorithm. The proposed algorithm demonstrates the credibility in terms of classification accuracy when compared to grid search techniques. Experimental results of the proposed algorithm also show promising performance in terms of computational speed.

[1]  Xiaoli Zhang,et al.  An ACO-based algorithm for parameter optimization of support vector machines , 2010, Expert Syst. Appl..

[2]  Krzysztof Socha,et al.  Ant Colony Optimisation for Continuous and Mixed-Variable Domains , 2009 .

[3]  Min Zhao,et al.  SBMDS: an interpretable string based malware detection system using SVM ensemble with bagging , 2009, Journal in Computer Virology.

[4]  Sheng Ding,et al.  Clonal Selection Algorithm for Feature Selection and Parameters Optimization of Support Vector Machines , 2009, 2009 Second International Symposium on Knowledge Acquisition and Modeling.

[5]  Christian Blum,et al.  Ant Colony Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[6]  Sreeram Ramakrishnan,et al.  A hybrid approach for feature subset selection using neural networks and ant colony optimization , 2007, Expert Syst. Appl..

[7]  A. Zell,et al.  Efficient parameter selection for support vector machines in classification and regression via model-based global optimization , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[8]  Marco Dorigo,et al.  Ant colony optimization for continuous domains , 2008, Eur. J. Oper. Res..

[9]  Xiaoli Zhang,et al.  A grid-based ACO algorithm for parameters optimization in support vector machines , 2008, 2008 IEEE International Conference on Granular Computing.

[10]  Zhonghang Xia,et al.  An optimization method for selecting parameters in support vector machines , 2007, Sixth International Conference on Machine Learning and Applications (ICMLA 2007).

[11]  Tao Bai,et al.  Share Price Prediction Using Wavelet Transform and Ant Colony Algorithm for Parameters Optimization in SVM , 2009, 2009 WRI Global Congress on Intelligent Systems.

[12]  Ioannis B. Theocharis,et al.  SVM-FuzCoC: A novel SVM-based feature selection method using a fuzzy complementary criterion , 2010, Pattern Recognit..

[13]  Manuel López-Ibáñez,et al.  Ant colony optimization , 2010, GECCO '10.

[14]  Mohamed Cheriet,et al.  Optimizing resources in model selection for support vector machines , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[15]  K. Lebart,et al.  A stochastic optimization approach for parameter tuning of support vector machines , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[16]  Yingqin Zhang.  Evolutionary Computation Based Automatic SVM Model Selection , 2008, 2008 Fourth International Conference on Natural Computation.

[17]  Wei Li,et al.  Three-class classification models of logS and logP derived by using GA–CG–SVM approach , 2009, Molecular Diversity.

[18]  Cheng-Lung Huang,et al.  ACO-based hybrid classification system with feature subset selection and model parameters optimization , 2009, Neurocomputing.

[19]  E. Alba,et al.  Metaheuristic Procedures for Training Neutral Networks , 2006 .

[20]  Ching Y. Suen,et al.  Automatic model selection for the optimization of SVM kernels , 2005, Pattern Recognit..

[21]  Cheng-Lung Huang,et al.  A GA-based feature selection and parameters optimizationfor support vector machines , 2006, Expert Syst. Appl..

[22]  Himanshu Gupta,et al.  Transistor size optimization in digital circuits using ant colony optimization for continuous domain , 2014, Int. J. Circuit Theory Appl..

[23]  Christian Blum,et al.  Ant colony optimization: Introduction and recent trends , 2005 .

[24]  Hongtao Zhang,et al.  Feature Selection for the Stored-grain Insects Based on PSO and SVM , 2009, 2009 Second International Workshop on Knowledge Discovery and Data Mining.

[25]  Vincent S. Tseng,et al.  Effective temporal data classification by integrating sequential pattern mining and probabilistic induction , 2009, Expert Syst. Appl..

[26]  S.R. Abbas,et al.  Electric Load Forecasting Using Support Vector Machines Optimized by Genetic Algorithm , 2006, 2006 IEEE International Multitopic Conference.

[27]  Cheng-Lung Huang,et al.  A distributed PSO-SVM hybrid system with feature selection and parameter optimization , 2008, Appl. Soft Comput..

[28]  Dexian Zhang,et al.  Feature Subset Selection Based on Improved Discrete Particle Swarm and Support Vector Machine Algorithm , 2009, 2009 International Conference on Information Engineering and Computer Science.

[29]  Mathias M. Adankon,et al.  Optimizing resources in model selection for support vector machines , 2005 .

[30]  Hasan Bal,et al.  Comparing performances of backpropagation and genetic algorithms in the data classification , 2011, Expert Syst. Appl..