Optimal feature selection using distance-based discrete firefly algorithm with mutual information criterion

In this paper, we investigate feature subset selection problem by a new self-adaptive firefly algorithm (FA), which is denoted as DbFAFS. In classical FA, it uses constant control parameters to solve different problems, which results in the premature of FA and the fireflies to be trapped in local regions without potential ability to explore new search space. To conquer the drawbacks of FA, we introduce two novel parameter selection strategies involving the dynamical regulation of the light absorption coefficient and the randomization control parameter. Additionally, as an important issue of feature subset selection problem, the objective function has a great effect on the selection of features. In this paper, we propose a criterion based on mutual information, and the criterion can not only measure the correlation between two features selected by a firefly but also determine the emendation of features among the achieved feature subset. The proposed approach is compared with differential evolution, genetic algorithm, and two versions of particle swarm optimization algorithm on several benchmark datasets. The results demonstrate that the proposed DbFAFS is efficient and competitive in both classification accuracy and computational performance.

[1]  Adel Al-Jumaily,et al.  Feature subset selection using differential evolution and a statistical repair mechanism , 2011, Expert Syst. Appl..

[2]  Andrés Iglesias,et al.  New memetic self-adaptive firefly algorithm for continuous optimisation , 2016 .

[3]  Xin-She Yang,et al.  Firefly algorithm with chaos , 2013, Commun. Nonlinear Sci. Numer. Simul..

[4]  Xin-She Yang,et al.  Firefly algorithm, stochastic test functions and design optimisation , 2010, Int. J. Bio Inspired Comput..

[5]  Farookh Khadeer Hussain,et al.  Support vector regression with chaos-based firefly algorithm for stock market price forecasting , 2013, Appl. Soft Comput..

[6]  Filiberto Pla,et al.  Supervised feature selection by clustering using conditional mutual information-based distances , 2010, Pattern Recognit..

[7]  A. Gandomi,et al.  Mixed variable structural optimization using Firefly Algorithm , 2011 .

[8]  Xin-She Yang,et al.  Cuckoo Search via Lévy flights , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).

[9]  Hossein Nezamabadi-pour,et al.  GSA: A Gravitational Search Algorithm , 2009, Inf. Sci..

[10]  Richard Nock,et al.  A hybrid filter/wrapper approach of feature selection using information theory , 2002, Pattern Recognit..

[11]  Milan Tuba,et al.  Firefly Algorithm for Cardinality Constrained Mean-Variance Portfolio Optimization Problem with Entropy Diversity Constraint , 2014, TheScientificWorldJournal.

[12]  Ming-Huwi Horng,et al.  Vector quantization using the firefly algorithm for image compression , 2012, Expert Syst. Appl..

[13]  Janez Brest,et al.  Memetic Self-Adaptive Firefly Algorithm , 2013 .

[14]  Chong-Ho Choi,et al.  Input feature selection for classification problems , 2002, IEEE Trans. Neural Networks.

[15]  James Kennedy,et al.  Defining a Standard for Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[16]  Jan M. Van Campenhout,et al.  On the Possible Orderings in the Measurement Selection Problem , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[17]  Randy L. Haupt,et al.  Practical Genetic Algorithms , 1998 .

[18]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[19]  Mohammad Kazem Sayadi,et al.  Firefly-inspired algorithm for discrete optimization problems: An application to manufacturing cell formation , 2013 .

[20]  Iztok Fister,et al.  Memetic firefly algorithm for combinatorial optimization , 2012, 1204.5165.

[21]  Christine M. Anderson-Cook Practical Genetic Algorithms (2nd ed.) , 2005 .

[22]  Thomas M. Cover,et al.  Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing) , 2006 .

[23]  M. Clerc Standard Particle Swarm Optimisation From 2006 to 2011 , 2012 .

[24]  Slawomir Zak,et al.  Firefly Algorithm for Continuous Constrained Optimization Tasks , 2009, ICCCI.

[25]  V. Mani,et al.  Clustering using firefly algorithm: Performance study , 2011, Swarm Evol. Comput..

[26]  Hossein Nezamabadi-pour,et al.  BGSA: binary gravitational search algorithm , 2010, Natural Computing.

[27]  Amir Hossein Gandomi,et al.  Firefly Algorithm for solving non-convex economic dispatch problems with valve loading effect , 2012, Appl. Soft Comput..

[28]  Patrick R. McMullen,et al.  Swarm intelligence: power in numbers , 2002, CACM.

[29]  Francisco José Madrid-Cuevas,et al.  Characterization of empirical discrepancy evaluation measures , 2004, Pattern Recognit. Lett..

[30]  Li-Yeh Chuang,et al.  Improved binary PSO for feature selection using gene expression data , 2008, Comput. Biol. Chem..

[31]  Ahmed Al-Ani,et al.  Feature Subset Selection Using Ant Colony Optimization , 2008 .

[32]  Ming-Feng Yeh,et al.  Grey particle swarm optimization , 2012, Appl. Soft Comput..

[33]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[34]  Janez Brest,et al.  A comprehensive review of firefly algorithms , 2013, Swarm Evol. Comput..

[35]  Andrés Iglesias,et al.  Firefly Algorithm for Polynomial Bézier Surface Parameterization , 2013, J. Appl. Math..

[36]  Erik D. Goodman,et al.  Swarmed feature selection , 2004, 33rd Applied Imagery Pattern Recognition Workshop (AIPR'04).

[37]  Leandro dos Santos Coelho,et al.  A chaotic firefly algorithm applied to reliability-redundancy optimization , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[38]  Sri Niwas Singh,et al.  Bacteria Foraging Optimization Based Bidding Strategy Under Transmission Congestion , 2015, IEEE Systems Journal.

[39]  Xin-She Yang,et al.  Firefly Algorithms for Multimodal Optimization , 2009, SAGA.

[40]  Xiaoming Liu,et al.  Mass Classification in Mammograms Using Selected Geometry and Texture Features, and a New SVM-Based Feature Selection Method , 2014, IEEE Systems Journal.

[41]  Xin-She Yang,et al.  Multiobjective firefly algorithm for continuous optimization , 2012, Engineering with Computers.

[42]  Gary Geunbae Lee,et al.  Information gain and divergence-based feature selection for machine learning-based text categorization , 2006, Inf. Process. Manag..

[43]  Huan Liu,et al.  Consistency-based search in feature selection , 2003, Artif. Intell..