Bare bones particle swarm optimization with adaptive chaotic jump for feature selection in classification

Feature selection (FS) is a crucial data pre-processing process in classification problems. It aims to reduce the dimensionality of the problem by eliminating irrelevant or redundant features while achieve similar or even higher classification accuracy than using all the features. As a variant of particle swarm optimization (PSO), Bare bones particle swarm optimization (BBPSO) is a simple but very powerful optimizer. However, it also suffers from premature convergence like other PSO algorithms, especially in high-dimensional optimization problems. In order to improve its performance in FS problems, this paper proposes a novel BBPSO based FS method called BBPSOACJ. An adaptive chaotic jump strategy is designed to help the stagnated particles make a large change in their searching trajectory. It can enrich the search behavior of BBPSO and prevent the particles from being trapped into local attractors. A new global best updating mechanism is employed to reduce the size of obtained feature subset. The proposed BBPSO-ACJ is compared with eight evolutionary computation (EC) based wrapper methods and two filter methods on nine benchmark datasets with different number of dimensions and instances. The experimenta l results indicate that the proposed method can select the most discriminative features from the entire feature set and achieve significantly better classification performance than other comparative methods.

[1]  Kazuyuki Murase,et al.  A new local search based hybrid genetic algorithm for feature selection , 2011, Neurocomputing.

[2]  Jong-Bae Park,et al.  A New Quantum-Inspired Binary PSO: Application to Unit Commitment Problems for Power Systems , 2010, IEEE Transactions on Power Systems.

[3]  James Kennedy,et al.  Bare bones particle swarms , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[4]  Feng Chu,et al.  A General Wrapper Approach to Selection of Class-Dependent Features , 2008, IEEE Transactions on Neural Networks.

[5]  Dixiong Yang,et al.  Chaotic Characteristic Analysis of Strong Earthquake Ground Motions , 2012, Int. J. Bifurc. Chaos.

[6]  Brian J. d'Auriol,et al.  A novel feature selection method based on normalized mutual information , 2011, Applied Intelligence.

[7]  M. Friedman A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[8]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[9]  Tae-Yong Choi,et al.  Experimental results of heterogeneous cooperative Bare Bones Particle Swarm Optimization with Gaussian jump for large scale global optimization , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[10]  Sheng Ding,et al.  Feature Selection Based F-Score and ACO Algorithm in Support Vector Machine , 2009, 2009 Second International Symposium on Knowledge Acquisition and Modeling.

[11]  Bo Liu,et al.  Improved particle swarm optimization combined with chaos , 2005 .

[12]  Li-Yeh Chuang,et al.  Chaotic maps based on binary particle swarm optimization for feature selection , 2011, Appl. Soft Comput..

[13]  Ashish Ghosh,et al.  Self-adaptive differential evolution for feature selection in hyperspectral image data , 2013, Appl. Soft Comput..

[14]  Keiji Tatsumi,et al.  A chaotic particle swarm optimization exploiting a virtual quartic objective function based on the personal and global best solutions , 2013, Appl. Math. Comput..

[15]  Parham Moradi,et al.  A hybrid particle swarm optimization for feature subset selection by integrating a novel local search strategy , 2016, Appl. Soft Comput..

[16]  Dun-Wei Gong,et al.  Feature selection algorithm based on bare bones particle swarm optimization , 2015, Neurocomputing.

[17]  A. Gandomi,et al.  Imperialist competitive algorithm combined with chaos for global optimization , 2012 .

[18]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[19]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[20]  Andries Petrus Engelbrecht,et al.  Bare bones differential evolution , 2009, Eur. J. Oper. Res..

[21]  B. Alatas Uniform Big Bang–Chaotic Big Crunch optimization , 2011 .

[22]  Li-Yeh Chuang,et al.  Improved binary particle swarm optimization using catfish effect for feature selection , 2011, Expert Syst. Appl..

[23]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[24]  Chao-Ton Su,et al.  Applying electromagnetism-like mechanism for feature selection , 2011, Inf. Sci..

[25]  Xiangyang Wang,et al.  Feature selection based on rough sets and particle swarm optimization , 2007, Pattern Recognit. Lett..

[26]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[27]  Leslie S. Smith,et al.  Feature subset selection in large dimensionality domains , 2010, Pattern Recognit..

[28]  Mengjie Zhang,et al.  Particle swarm optimisation for feature selection: A hybrid filter-wrapper approach , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[29]  Nasser Ghasem-Aghaee,et al.  Text feature selection using ant colony optimization , 2009, Expert Syst. Appl..

[30]  Mengjie Zhang,et al.  Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms , 2014, Appl. Soft Comput..

[31]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[32]  Mengjie Zhang,et al.  A PSO based hybrid feature selection algorithm for high-dimensional classification , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[33]  B. Alatas,et al.  Chaos embedded particle swarm optimization algorithms , 2009 .

[34]  Hao Gao,et al.  Adaptive bare bones particle swarm optimization for feature selection , 2016, 2016 Chinese Control and Decision Conference (CCDC).

[35]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[36]  Chuyi Song,et al.  Hybrid Algorithm Based on Particle Swarm Optimization and Artificial Fish Swarm Algorithm , 2012, ISNN.

[37]  Hao Liu,et al.  Bare-bones particle swarm optimization with disruption operator , 2014, Appl. Math. Comput..

[38]  LeeSungyoung,et al.  A novel feature selection method based on normalized mutual information , 2012 .

[39]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[40]  Tim Blackwell,et al.  A Study of Collapse in Bare Bones Particle Swarm Optimization , 2012, IEEE Transactions on Evolutionary Computation.