Multi-objective Feature Selection in Classification: A Differential Evolution Approach

Feature selection is an important pre-processing step in classification tasks. Feature selection aims to minimise both the classification error rate and the number of features, which are usually two conflicting objectives. This paper develops a differential evolution DE based multi-objective feature selection approach. The multi-objective approach is compared with two conventional methods and two DE based single objective methods, where the first algorithm is to minimise the classification error rate only while the second algorithm combines the number of features and the classification error rate into a single fitness function. Their performances are examined on nine different datasets and the results show that the proposed multi-objective algorithm successfully evolved a number of trade-off solutions, which reduce the number of features and keep or reduce the classification error rate. In almost all cases, the proposed multi-objective algorithm achieved better performance than all the other four methods in terms of both the classification accuracy and the number of features.

[1]  Duoqian Miao,et al.  A rough set approach to feature selection based on ant colony optimization , 2010, Pattern Recognit. Lett..

[2]  Adel M. Alimi,et al.  Distributed MOPSO with a new population subdivision technique for the feature selection , 2011, 2011 5th International Symposium on Computational Intelligence and Intelligent Informatics (ISCIII).

[3]  Mengjie Zhang,et al.  A multi-objective particle swarm optimisation for filter-based feature selection in classification problems , 2012, Connect. Sci..

[4]  Mengjie Zhang,et al.  Differential evolution (DE) for multi-objective feature selection in classification , 2014, GECCO.

[5]  Mengjie Zhang,et al.  Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach , 2013, IEEE Transactions on Cybernetics.

[6]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation) , 2006 .

[7]  Mengjie Zhang,et al.  Novel Initialisation and Updating Mechanisms in PSO for Feature Selection in Classification , 2013, EvoApplications.

[8]  Mengjie Zhang,et al.  Pareto front feature selection: using genetic programming to explore feature space , 2009, GECCO.

[9]  Panos M. Pardalos,et al.  Feature selection based on meta-heuristics for biomedicine , 2014, Optim. Methods Softw..

[10]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[11]  Yan Dong,et al.  Feature Selection with Discrete Binary Differential Evolution , 2009, 2009 International Conference on Artificial Intelligence and Computational Intelligence.

[12]  Mengjie Zhang,et al.  Binary particle swarm optimisation for feature selection: A filter based approach , 2012, 2012 IEEE Congress on Evolutionary Computation.

[13]  Mengjie Zhang,et al.  Multi-objective particle swarm optimisation (PSO) for feature selection , 2012, GECCO '12.

[14]  Bogdan Filipic,et al.  DEMO: Differential Evolution for Multiobjective Optimization , 2005, EMO.

[15]  Eibe Frank,et al.  Large-scale attribute selection using wrappers , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[16]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[17]  Mengjie Zhang,et al.  Binary PSO and Rough Set Theory for Feature Selection: a Multi-objective filter Based Approach , 2014, Int. J. Comput. Intell. Appl..

[18]  Fakhri Karray,et al.  Multi-objective Feature Selection with NSGA II , 2007, ICANNGA.

[19]  R. Storn,et al.  On the usage of differential evolution for function optimization , 1996, Proceedings of North American Fuzzy Information Processing.

[20]  Mengjie Zhang,et al.  Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms , 2014, Appl. Soft Comput..

[21]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[22]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[23]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[24]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[25]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[26]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[27]  Rami N. Khushaba,et al.  Feature subset selection using differential evolution and a wheel based search strategy , 2013, Swarm Evol. Comput..

[28]  Zexuan Zhu,et al.  Wrapper–Filter Feature Selection Algorithm Using a Memetic Framework , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[29]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..