Many-objective optimization of feature selection based on two-level particle cooperation

Abstract Feature selection (FS) plays a crucial role in classification, which aims to remove redundant and irrelevant data features.unknown However, for high-dimensional FS problems, Pareto optimal solutions are usually sparse, signifying that most of the decision variables are zero. Solving such problems using most existing evolutionary algorithms is difficult. In this paper, we reformulate FS as a many-objective optimization problem comprising three objectives to be minimized. To solve this problem, we proposed a binary particle swarm optimization with a two-level particle cooperation strategy. In the first level, to maintain rapid convergence, randomly generated ordinary particles and strict particles filtered by ReliefF are combined as the initialized particles. In the second level, under the decomposition multiobjective optimization framework, cooperation between particles is conducted during the update process to search for Pareto solutions more efficiently. In addition, a many-objective reset operation is proposed to enable the algorithm to jump out of the local optimum. Experimental studies on 10 real-world benchmark data sets revealed that our proposed algorithm could effectively reduce the number of features and achieve a competitive classification accuracy compared with some state-of-the-art evolutionary FS methods and non-evolutionary approaches.

[1]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[2]  Qingfu Zhang,et al.  A Two-Phase Evolutionary Approach for Compressive Sensing Reconstruction , 2017, IEEE Transactions on Cybernetics.

[3]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Maoguo Gong,et al.  A Multiobjective Cooperative Coevolutionary Algorithm for Hyperspectral Sparse Unmixing , 2017, IEEE Transactions on Evolutionary Computation.

[5]  Dunwei Gong,et al.  Binary differential evolution with self-learning for multi-objective feature selection , 2020, Inf. Sci..

[6]  Aruna Tiwari,et al.  Construction of classifier with feature selection based on genetic programming , 2010, IEEE Congress on Evolutionary Computation.

[7]  Alper Ekrem Murat,et al.  A discrete particle swarm optimization method for feature selection in binary classification problems , 2010, Eur. J. Oper. Res..

[8]  Xiangyang Wang,et al.  Feature selection based on rough sets and particle swarm optimization , 2007, Pattern Recognit. Lett..

[9]  James Kennedy,et al.  Bare bones particle swarms , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[10]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[11]  Mengjie Zhang,et al.  Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach , 2013, IEEE Transactions on Cybernetics.

[12]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[13]  Maoguo Gong,et al.  A Multiobjective Sparse Feature Learning Model for Deep Neural Networks , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Xin Yao,et al.  A Survey on Evolutionary Computation Approaches to Feature Selection , 2016, IEEE Transactions on Evolutionary Computation.

[15]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[16]  Mengjie Zhang,et al.  Pareto front feature selection based on artificial bee colony optimization , 2018, Inf. Sci..

[17]  Li-Yeh Chuang,et al.  Improved binary PSO for feature selection using gene expression data , 2008, Comput. Biol. Chem..

[18]  Jian Cheng,et al.  Multi-Objective Particle Swarm Optimization Approach for Cost-Based Feature Selection in Classification , 2017, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[19]  Verónica Bolón-Canedo,et al.  A review of microarray datasets and applied feature selection methods , 2014, Inf. Sci..

[20]  John E. Dennis,et al.  Normal-Boundary Intersection: A New Method for Generating the Pareto Surface in Nonlinear Multicriteria Optimization Problems , 1998, SIAM J. Optim..

[21]  Anirban Mukhopadhyay,et al.  A Graph-Theoretic Approach for Identifying Non-Redundant and Relevant Gene Markers from Microarray Data Using Multiobjective Binary PSO , 2014, PloS one.

[22]  Mengjie Zhang,et al.  Multi-objective particle swarm optimisation (PSO) for feature selection , 2012, GECCO '12.

[23]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[24]  Mengjie Zhang,et al.  A New Representation in PSO for Discretisation-Based Feature Selection , 2017 .

[25]  Wlodzislaw Duch,et al.  Feature Selection for High-Dimensional Data - A Pearson Redundancy Based Filter , 2008, Computer Recognition Systems 2.

[26]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[27]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[28]  Mohammad Karim Sohrabi,et al.  Multi-objective feature selection for warfarin dose prediction , 2017, Comput. Biol. Chem..

[29]  Qingfu Zhang,et al.  Stable Matching-Based Selection in Evolutionary Multiobjective Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[30]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[31]  Mengjie Zhang,et al.  A Multi-objective Feature Selection Approach Based on Binary PSO and Rough Set Theory , 2013, EvoCOP.

[32]  R. Lyndon While,et al.  A faster algorithm for calculating hypervolume , 2006, IEEE Transactions on Evolutionary Computation.

[33]  Mengjie Zhang,et al.  A multi-objective particle swarm optimisation for filter-based feature selection in classification problems , 2012, Connect. Sci..

[34]  Zongben Xu,et al.  A multiobjective ACO algorithm for rough feature selection , 2010, 2010 Second Pacific-Asia Conference on Circuits, Communications and System.

[35]  Swagatam Das,et al.  A closed loop stability analysis and parameter selection of the Particle Swarm Optimization dynamics for faster convergence , 2007, 2007 IEEE Congress on Evolutionary Computation.

[36]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[37]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[38]  Yu Xue,et al.  Text classification based on deep belief network and softmax regression , 2016, Neural Computing and Applications.

[39]  Chaoyong Zhang,et al.  Selective cooperative disassembly planning based on multi-objective discrete artificial bee colony algorithm , 2017, Eng. Appl. Artif. Intell..