A memetic algorithm with support vector machine for feature selection and classification

The memetic algorithm (MA) is an evolutionary metaheuristic that can be viewed as a hybrid genetic algorithm combined with some kinds of local search. In this paper, we propose a memetic algorithm combined with a support vector machine (SVM) for feature selection and classification in Data mining. The proposed approach tries to find a subset of features that maximizes the classification accuracy rate of SVM. In addition, another hybrid algorithm of MA and SVM with optimized parameters is also developed. The two versions of our proposed method are evaluated on some datasets and compared with some well-known classifiers for data classification. The computational experiments show that the hybrid method MA + SVM with optimized parameters provides competitive results and finds high quality solutions.

[1]  Rich Caruana,et al.  Greedy Attribute Selection , 1994, ICML.

[2]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[3]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[4]  Habiba Drias,et al.  Local Search Methods for the Optimal Winner Determination Problem in Combinatorial Auctions , 2010, J. Math. Model. Algorithms.

[5]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[6]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[7]  Zhongyi Hu,et al.  A PSO and pattern search based memetic algorithm for SVMs parameters optimization , 2013, Neurocomputing.

[8]  R. Venkata Rao,et al.  Teaching-Learning-Based Optimization: An optimization method for continuous non-linear large scale problems , 2012, Inf. Sci..

[9]  Kenneth A. De Jong,et al.  Measurement of Population Diversity , 2001, Artificial Evolution.

[10]  Kay Chen Tan,et al.  A Multi-Facet Survey on Memetic Computation , 2011, IEEE Transactions on Evolutionary Computation.

[11]  Kay Chen Tan,et al.  A hybrid evolutionary algorithm for attribute selection in data mining , 2009, Expert Syst. Appl..

[12]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[13]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[14]  H. Drias,et al.  Solving MAX-SAT problems using a memetic evolutionary meta-heuristic , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[15]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[16]  Sven F. Crone,et al.  Genetic Algorithms for Support Vector Machine Model Selection , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[17]  Dalila Boughaci,et al.  Improving Support Vector Machine Using a Stochastic Local Search for Classification in DataMining , 2012, ICONIP.

[18]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[19]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[20]  Lutz Hamel,et al.  Knowledge Discovery with Support Vector Machines , 2009 .

[21]  Vojislav Kecman,et al.  Learning and Soft Computing: Support Vector Machines, Neural Networks, and Fuzzy Logic Models , 2001 .

[22]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[23]  Ian H. Witten,et al.  Generating Accurate Rule Sets Without Global Optimization , 1998, ICML.

[24]  Jin-Kao Hao,et al.  A Hybrid GA/SVM Approach for Gene Selection and Classification of Microarray Data , 2006, EvoWorkshops.

[25]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[26]  Pablo Moscato,et al.  On Evolution, Search, Optimization, Genetic Algorithms and Martial Arts : Towards Memetic Algorithms , 1989 .

[27]  Bu-Sung Lee,et al.  Memetic algorithm using multi-surrogates for computationally expensive optimization problems , 2007, Soft Comput..

[28]  Xin Yao,et al.  A Memetic Algorithm for VLSI Floorplanning , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[29]  Ying Li,et al.  An Improved Particle Swarm Optimization for SVM Training , 2007, Third International Conference on Natural Computation (ICNC 2007).

[30]  Xiao Zhi Gao,et al.  A memetic-inspired harmony search method in optimal wind generator design , 2015, Int. J. Mach. Learn. Cybern..

[31]  Colin Campbell,et al.  Learning with Support Vector Machines , 2011, Learning with Support Vector Machines.

[32]  Habiba Drias,et al.  A memetic algorithm for the optimal winner determination problem , 2009, Soft Comput..

[33]  Mark Sumner,et al.  A Fast Adaptive Memetic Algorithm for Online and Offline Control Design of PMSM Drives , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[34]  Jing Tang,et al.  Diversity-adaptive parallel memetic algorithm for solving large scale combinatorial optimization problems , 2006, Soft Comput..