Particle Swarm Optimisation with genetic operators for feature selection

Feature selection is an important task in machine learning, which aims to reduce the dataset dimensionality while at least maintaining the classification performance. Particle Swarm Optimisation (PSO) has been widely applied to feature selection because of its effectiveness and efficiency. However, since feature selection is a challenging task with a complex search space, PSO easily gets stuck at local optima. This paper aims to improve the PSO's searching ability by applying genetic operators such as crossover and mutation to assist the swarm to explore the search space better. The proposed genetic operators are specifically designed for feature selection, which not only improve the quality of current feature subsets but also make the search smoother. The proposed algorithm, called CMPSO, is tested and compared with three recent PSO based feature selection algorithms. Experimental results on eight datasets show that CMPSO can adapt with different numbers of features to evolve small feature subsets, which achieve similar or better classification performance than using all features and the three PSO based algorithms. The analysis on evolutionary processes shows that genetic operators assist CMPSO to evolve better solutions than the original PSO.

[1]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[2]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[3]  João Miguel da Costa Sousa,et al.  Modified binary PSO for feature selection using SVM applied to mortality prediction of septic patients , 2013, Appl. Soft Comput..

[4]  Mengjie Zhang,et al.  A multi-objective particle swarm optimisation for filter-based feature selection in classification problems , 2012, Connect. Sci..

[5]  Yudong Zhang,et al.  Binary PSO with mutation operator for feature selection using decision tree applied to spam detection , 2014, Knowl. Based Syst..

[6]  Pramod Kumar Singh,et al.  Opposition chaotic fitness mutation based adaptive inertia weight BPSO for feature selection in text clustering , 2016, Appl. Soft Comput..

[7]  A. Wayne Whitney,et al.  A Direct Method of Nonparametric Measurement Selection , 1971, IEEE Transactions on Computers.

[8]  Enrique Alba,et al.  Gene selection in cancer classification using PSO/SVM and GA/SVM hybrid algorithms , 2007, 2007 IEEE Congress on Evolutionary Computation.

[9]  Russell C. Eberhart,et al.  Comparison between Genetic Algorithms and Particle Swarm Optimization , 1998, Evolutionary Programming.

[10]  Mengjie Zhang,et al.  Multi-objective particle swarm optimisation (PSO) for feature selection , 2012, GECCO '12.

[11]  Teresa Wu,et al.  An Adaptive Particle Swarm Optimization With Multiple Adaptive Methods , 2013, IEEE Transactions on Evolutionary Computation.

[12]  Muhammad Nazir,et al.  PSO-GA Based Optimized Feature Selection Using Facial and Clothing Information for Gender Classification , 2014 .

[13]  Xin Yao,et al.  A Survey on Evolutionary Computation Approaches to Feature Selection , 2016, IEEE Transactions on Evolutionary Computation.

[14]  Mengjie Zhang,et al.  PSO and Statistical Clustering for Feature Selection: A New Representation , 2014, SEAL.

[15]  Lloyd A. Smith,et al.  Feature Selection for Machine Learning: Comparing a Correlation-Based Filter Approach to the Wrapper , 1999, FLAIRS.

[16]  Mengjie Zhang,et al.  Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms , 2014, Appl. Soft Comput..

[17]  Ping Wang,et al.  A new feature selection method based on relief and SVM-RFE , 2014, 2014 12th International Conference on Signal Processing (ICSP).

[18]  Bing Xue,et al.  Mutual Information Estimation for Filter Based Feature Selection Using Particle Swarm Optimization , 2016, EvoApplications.

[19]  Mengjie Zhang,et al.  Gaussian Transformation Based Representation in Particle Swarm Optimisation for Feature Selection , 2015, EvoApplications.

[20]  Li-Yeh Chuang,et al.  Improved binary PSO for feature selection using gene expression data , 2008, Comput. Biol. Chem..

[21]  Jiawei Han,et al.  Generalized Fisher Score for Feature Selection , 2011, UAI.

[22]  Mengjie Zhang,et al.  Gaussian Based Particle Swarm Optimisation and Statistical Clustering for Feature Selection , 2014, EvoCOP.

[23]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[24]  Jon Atli Benediktsson,et al.  Feature Selection Based on Hybridization of Genetic Algorithm and Particle Swarm Optimization , 2015, IEEE Geoscience and Remote Sensing Letters.

[25]  David Casasent,et al.  An improvement on floating search algorithms for feature subset selection , 2009, Pattern Recognit..

[26]  Yudong Zhang,et al.  A Comprehensive Survey on Particle Swarm Optimization Algorithm and Its Applications , 2015 .

[27]  Shutao Li,et al.  Gene selection using hybrid particle swarm optimization and genetic algorithm , 2008, Soft Comput..

[28]  Xianhua Li,et al.  Band selection based on evolution algorithm and sequential search for hyperspectral classification , 2008, 2008 International Conference on Audio, Language and Image Processing.