A fast wrapper feature subset selection method based on binary particle swarm optimization

Although many particle swarm optimization (PSO) based feature subset selection methods have been proposed, most of them seem to ignore the difference of feature subset selection problems and other optimization problems. We analyze the search process of a PSO based wrapper feature subset selection algorithm and find that characteristics of feature subset selection can be used to optimize this process. We compare wrapper and filter ways of evaluating features and define the domain knowledge of feature subset selection problems and we propose a fast wrapper feature subset selection algorithm based on PSO employed the domain knowledge of feature subset selection problems. Experimental results show that our method can work well, and the new algorithm can improve both the running time and the classification accuracy.

[1]  Xiangyang Wang,et al.  Feature selection based on rough sets and particle swarm optimization , 2007, Pattern Recognit. Lett..

[2]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[3]  Mengjie Zhang,et al.  Binary particle swarm optimisation for feature selection: A filter based approach , 2012, 2012 IEEE Congress on Evolutionary Computation.

[4]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[5]  Mengjie Zhang,et al.  Multi-objective particle swarm optimisation (PSO) for feature selection , 2012, GECCO '12.

[6]  B. Chakraborty.  Feature subset selection by particle swarm optimization with fuzzy fitness function , 2008, 2008 3rd International Conference on Intelligent System and Knowledge Engineering.

[7]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[8]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[9]  Thomas G. Dietterich,et al.  Learning with Many Irrelevant Features , 1991, AAAI.

[10]  Ron Kohavi,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998 .

[11]  Larry A. Rendell,et al.  The Feature Selection Problem: Traditional Methods and a New Algorithm , 1992, AAAI.

[12]  James Kennedy,et al.  Bare bones particle swarms , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[13]  Thomas G. Dietterich,et al.  Learning Boolean Concepts in the Presence of Many Irrelevant Features , 1994, Artif. Intell..

[14]  Chukwudi Anyakoha,et al.  A review of particle swarm optimization. Part I: background and development , 2007, Natural Computing.

[15]  Aiguo Li,et al.  Feature Subset Selection Based on Binary Particle Swarm Optimization and Overlap Information Entropy , 2009, 2009 International Conference on Computational Intelligence and Software Engineering.

[16]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[17]  Huan Liu,et al.  Efficient Feature Selection via Analysis of Relevance and Redundancy , 2004, J. Mach. Learn. Res..

[18]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[19]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[20]  Li-Yeh Chuang,et al.  Improved binary PSO for feature selection using gene expression data , 2008, Comput. Biol. Chem..

[21]  George D. C. Cavalcanti,et al.  An approach to feature selection for keystroke dynamics systems based on PSO and feature weighting , 2007, 2007 IEEE Congress on Evolutionary Computation.

[22]  J. Anuradha,et al.  A Survey on Particle Swarm Optimization in Feature Selection , 2011 .

[23]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[24]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[25]  Jun Wang,et al.  Effective feature selection with Particle Swarm Optimization based one-dimension searching , 2010, 2010 3rd International Symposium on Systems and Control in Aeronautics and Astronautics.

[26]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Chukwudi Anyakoha,et al.  A review of particle swarm optimization. Part II: hybridisation, combinatorial, multicriteria and constrained optimization, and indicative applications , 2008, Natural Computing.