Feature subset selection using differential evolution and a statistical repair mechanism

One of the fundamental motivations for feature selection is to overcome the curse of dimensionality problem. This paper presents a novel feature selection method utilizing a combination of differential evolution (DE) optimization method and a proposed repair mechanism based on feature distribution measures. The new method, abbreviated as DEFS, utilizes the DE float number optimizer in the combinatorial optimization problem of feature selection. In order to make the solutions generated by the float-optimizer suitable for feature selection, a roulette wheel structure is constructed and supplied with the probabilities of features distribution. These probabilities are constructed during iterations by identifying the features that contribute to the most promising solutions. The proposed DEFS is used to search for optimal subsets of features in datasets with varying dimensionality. It is then utilized to aid in the selection of Wavelet Packet Transform (WPT) best basis for classification problems, thus acting as a part of a feature extraction process. Practical results indicate the significance of the proposed method in comparison with other feature selection methods.

[1]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[2]  Karim Faez,et al.  GA-based optimal selection of PZMI features for face recognition , 2008, Appl. Math. Comput..

[3]  Jianjiang Lu,et al.  Feature selection based-on genetic algorithm for image annotation , 2008, Knowl. Based Syst..

[4]  Dobrivoje Popovic,et al.  Computational Intelligence in Time Series Forecasting: Theory and Engineering Applications (Advances in Industrial Control) , 2005 .

[5]  Thomas Stützle,et al.  Ant Colony Optimization Theory , 2004 .

[6]  Karim Faez,et al.  An improved feature selection method based on ant colony optimization (ACO) evaluated on face recognition system , 2008, Appl. Math. Comput..

[7]  Hiroshi Motoda,et al.  Computational Methods of Feature Selection , 2022 .

[8]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[9]  Chris H. Q. Ding,et al.  Evolving Feature Selection , 2005, IEEE Intell. Syst..

[10]  Anil K. Jain,et al.  Dimensionality reduction using genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[11]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[12]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[13]  Erik D. Goodman,et al.  Swarmed feature selection , 2004, 33rd Applied Imagery Pattern Recognition Workshop (AIPR'04).

[14]  Christine M. Anderson-Cook Practical Genetic Algorithms (2nd ed.) , 2005 .

[15]  Byung Ro Moon,et al.  Hybrid Genetic Algorithms for Feature Selection , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Li-Yeh Chuang,et al.  Improved binary PSO for feature selection using gene expression data , 2008, Comput. Biol. Chem..

[17]  Ahmed Al-Ani,et al.  Feature Subset Selection Using Ant Colony Optimization , 2008 .

[18]  Andrew K. Chan,et al.  Fundamentals of Wavelets: Theory, Algorithms, and Applications , 2011 .

[19]  Nasser Ghasem-Aghaee,et al.  Text feature selection using ant colony optimization , 2009, Expert Syst. Appl..

[20]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[21]  Qiang Shen,et al.  Computational Intelligence and Feature Selection - Rough and Fuzzy Approaches , 2008, IEEE Press series on computational intelligence.

[22]  Donald E. Brown,et al.  Fast generic selection of features for neural network classifiers , 1992, IEEE Trans. Neural Networks.

[23]  Randy L. Haupt,et al.  Practical Genetic Algorithms , 1998 .