Evolutionary Population Dynamics and Grasshopper Optimization approaches for feature selection problems

Abstract Searching for the optimal subset of features is known as a challenging problem in feature selection process. To deal with the difficulties involved in this problem, a robust and reliable optimization algorithm is required. In this paper, Grasshopper Optimization Algorithm (GOA) is employed as a search strategy to design a wrapper-based feature selection method. The GOA is a recent population-based metaheuristic that mimics the swarming behaviors of grasshoppers. In this work, an efficient optimizer based on the simultaneous use of the GOA, selection operators, and Evolutionary Population Dynamics (EPD) is proposed in the form of four different strategies to mitigate the immature convergence and stagnation drawbacks of the conventional GOA. In the first two approaches, one of the top three agents and a randomly generated one are selected to reposition a solution from the worst half of the population. In the third and fourth approaches, to give a chance to the low fitness solutions in reforming the population, Roulette Wheel Selection (RWS) and Tournament Selection (TS) are utilized to select the guiding agent from the first half. The proposed GOA_EPD approaches are employed to tackle various feature selection tasks. The proposed approaches are benchmarked on 22 UCI datasets. The comprehensive results and various comparisons reveal that the EPD has a remarkable impact on the efficacy of the GOA and using the selection mechanism enhanced the capability of the proposed approach to outperform other optimizers and find the best solutions with improved convergence trends. Furthermore, the comparative experiments demonstrate the superiority of the proposed approaches when compared to other similar methods in the literature.

[1]  Hossam Faris,et al.  Training feedforward neural networks using multi-verse optimizer for binary classification problems , 2016, Applied Intelligence.

[2]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[3]  Seyed Mohammad Mirjalili,et al.  Evolutionary population dynamics and grey wolf optimizer , 2015, Neural Computing and Applications.

[4]  Q. Henry Wu,et al.  Wind-thermal power system dispatch using MLSAD model and GSOICLW algorithm , 2017, Knowl. Based Syst..

[5]  Hossam Faris,et al.  Optimizing the Learning Process of Feedforward Neural Networks Using Lightning Search Algorithm , 2016, Int. J. Artif. Intell. Tools.

[6]  Aboul Ella Hassanien,et al.  MOGOA algorithm for constrained and unconstrained multi-objective optimization problems , 2017, Applied Intelligence.

[7]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[8]  Majdi M. Mafarja,et al.  Hybrid Whale Optimization Algorithm with simulated annealing for feature selection , 2017, Neurocomputing.

[9]  Gil Alterovitz,et al.  Accelerating wrapper-based feature selection with K-nearest-neighbor , 2015, Knowl. Based Syst..

[10]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[11]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[12]  David E. Goldberg,et al.  Genetic Algorithms, Tournament Selection, and the Effects of Noise , 1995, Complex Syst..

[13]  Ibrahim Aljarah,et al.  Parallel particle swarm optimization clustering algorithm based on MapReduce methodology , 2012, 2012 Fourth World Congress on Nature and Biologically Inspired Computing (NaBIC).

[14]  Hossam M. Zawbaa,et al.  Feature selection based on antlion optimization algorithm , 2015, 2015 Third World Conference on Complex Systems (WCCS).

[15]  Crina Grosan,et al.  Feature Subset Selection Approach by Gray-Wolf Optimization , 2014, AECIA.

[16]  Qiang Shen,et al.  Finding Rough Set Reducts with Ant Colony Optimization , 2003 .

[17]  Andrew Lewis,et al.  Evolutionary Population Dynamics and Multi-Objective Optimisation Problems , 2008 .

[18]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[19]  Crina Grosan,et al.  Feature Selection via Chaotic Antlion Optimization , 2016, PloS one.

[20]  Almoataz Y. Abdelaziz,et al.  Ant Lion Optimization Algorithm for optimal location and sizing of renewable distributed generations , 2017 .

[21]  Hossein Nezamabadi-pour,et al.  BGSA: binary gravitational search algorithm , 2010, Natural Computing.

[22]  David G. Stork,et al.  Pattern Classification , 1973 .

[23]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[24]  Ronghua Shang,et al.  Subspace learning-based graph regularized feature selection , 2016, Knowl. Based Syst..

[25]  Hossam Faris,et al.  Grasshopper optimization algorithm for multi-objective optimization problems , 2017, Applied Intelligence.

[26]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[27]  Salwani Abdullah,et al.  Investigating memetic algorithm in solving rough set attribute reduction , 2013, Int. J. Comput. Appl. Technol..

[28]  Tang,et al.  Self-Organized Criticality: An Explanation of 1/f Noise , 2011 .

[29]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[30]  Seyed Mohammad Mirjalili,et al.  The Ant Lion Optimizer , 2015, Adv. Eng. Softw..

[31]  Andrzej Skowron,et al.  Rough-Fuzzy Hybridization: A New Trend in Decision Making , 1999 .

[32]  Parham Pahlavani,et al.  An efficient modified grey wolf optimizer with Lévy flight for optimization tasks , 2017, Appl. Soft Comput..

[33]  Fei Wang,et al.  Fast semi-supervised clustering with enhanced spectral embedding , 2012, Pattern Recognit..

[34]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[35]  Stefan Boettcher,et al.  Extremal Optimization: Methods derived from Co-Evolution , 1999, GECCO.

[36]  Ronghua Shang,et al.  Non-Negative Spectral Learning and Sparse Regression-Based Dual-Graph Regularized Feature Selection , 2018, IEEE Transactions on Cybernetics.

[37]  Seyedali Mirjalili,et al.  SCA: A Sine Cosine Algorithm for solving optimization problems , 2016, Knowl. Based Syst..

[38]  N. Altman An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression , 1992 .

[39]  Haytham Elghazel,et al.  A semi-supervised feature ranking method with ensemble learning , 2012, Pattern Recognit. Lett..

[40]  Shi-Jinn Horng,et al.  Dynamic variable precision rough set approach for probabilistic set-valued information systems , 2017, Knowl. Based Syst..

[41]  Aboul Ella Hassanien,et al.  Binary grey wolf optimization approaches for feature selection , 2016, Neurocomputing.

[42]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[43]  Parham Moradi,et al.  Integration of graph clustering with ant colony optimization for feature selection , 2015, Knowl. Based Syst..

[44]  Selma Ayse Özel,et al.  A hybrid approach of differential evolution and artificial bee colony for feature selection , 2016, Expert Syst. Appl..

[45]  Xin-She Yang,et al.  Firefly Algorithms for Multimodal Optimization , 2009, SAGA.

[46]  Lloyd A. Smith,et al.  Feature Selection for Machine Learning: Comparing a Correlation-Based Filter Approach to the Wrapper , 1999, FLAIRS.

[47]  Andrew Lewis,et al.  The Whale Optimization Algorithm , 2016, Adv. Eng. Softw..

[48]  Radu-Emil Precup,et al.  Grey Wolf Optimizer Algorithm-Based Tuning of Fuzzy Control Systems With Reduced Parametric Sensitivity , 2017, IEEE Transactions on Industrial Electronics.

[49]  Salwani Abdullah,et al.  Fuzzy Population-Based Meta-Heuristic Approaches for Attribute Reduction in Rough Set Theory , 2015 .

[50]  Andrew Lewis,et al.  Grasshopper Optimisation Algorithm: Theory and application , 2017, Adv. Eng. Softw..

[51]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[52]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[53]  Yudong Zhang,et al.  Binary PSO with mutation operator for feature selection using decision tree applied to spam detection , 2014, Knowl. Based Syst..

[54]  Marcel J. T. Reinders,et al.  Random subspace method for multivariate feature selection , 2006, Pattern Recognit. Lett..

[55]  Keiichi Tamura,et al.  Distributed Modified Extremal Optimization using Island Model for Reducing Crossovers in Reconciliation Graph , 2013 .

[56]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[57]  Honglun Wang,et al.  Distributed trajectory optimization for multiple solar-powered UAVs target tracking in urban environment by Adaptive Grasshopper Optimization Algorithm , 2017 .

[58]  Ben Niu,et al.  A discrete bacterial algorithm for feature selection in classification of microarray gene expression cancer data , 2017, Knowl. Based Syst..

[59]  A. Rezaee Jordehi,et al.  An efficient chaotic water cycle algorithm for optimization tasks , 2015, Neural Computing and Applications.

[60]  Hao Chen,et al.  A Heuristic Feature Selection Approach for Text Categorization by Using Chaos Optimization and Genetic Algorithm , 2013 .

[61]  William Eberle,et al.  Genetic algorithms in feature and instance selection , 2013, Knowl. Based Syst..

[62]  Andrew Lewis,et al.  Extremal Optimisation for Assignment Type Problems , 2009 .

[63]  Yangyang Li,et al.  Self-representation based dual-graph regularized feature selection clustering , 2016, Neurocomputing.

[64]  Andrew Lewis,et al.  An Evolutionary Programming Algorithm for Automatic Engineering Design , 2003, PPAM.

[65]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[66]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[67]  Andrew Lewis,et al.  S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization , 2013, Swarm Evol. Comput..

[68]  Witold Pedrycz,et al.  An overview on the roles of fuzzy set techniques in big data processing: Trends, challenges and opportunities , 2017, Knowl. Based Syst..

[69]  Huan Liu,et al.  Neural-network feature selector , 1997, IEEE Trans. Neural Networks.

[70]  Attia A. El-Fergany,et al.  Electrical characterisation of proton exchange membrane fuel cells stack using grasshopper optimiser , 2018 .

[71]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[72]  Verónica Bolón-Canedo,et al.  Recent advances and emerging challenges of feature selection in the context of big data , 2015, Knowl. Based Syst..

[73]  Jack Sklansky,et al.  On Automatic Feature Selection , 1988, Int. J. Pattern Recognit. Artif. Intell..

[74]  Philip S. Yu,et al.  Forward Semi-supervised Feature Selection , 2008, PAKDD.

[75]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[76]  Zbigniew Michalewicz,et al.  Handbook of Evolutionary Computation , 1997 .

[77]  P. Langley Selection of Relevant Features in Machine Learning , 1994 .

[78]  Yu Huang,et al.  Path planning for solar-powered UAV in urban environment , 2018, Neurocomputing.

[79]  Hossam Faris,et al.  Optimizing connection weights in neural networks using the whale optimization algorithm , 2016, Soft Computing.

[80]  Xin-She Yang,et al.  Binary bat algorithm , 2013, Neural Computing and Applications.

[81]  Hossein Nezamabadi-pour,et al.  An advanced ACO algorithm for feature subset selection , 2015, Neurocomputing.

[82]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[83]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[84]  Aboul Ella Hassanien,et al.  Binary ant lion approaches for feature selection , 2016, Neurocomputing.

[85]  Qiang Shen,et al.  Semantics-preserving dimensionality reduction: rough and fuzzy-rough-based approaches , 2004, IEEE Transactions on Knowledge and Data Engineering.

[86]  Hossam Faris,et al.  Grey wolf optimizer: a review of recent variants and applications , 2017, Neural Computing and Applications.

[87]  Hossam Faris,et al.  Improved monarch butterfly optimization for unconstrained global search and neural network training , 2018, Applied Intelligence.

[88]  Kup-Sze Choi,et al.  Synchronization clustering based on central force optimization and its extension for large-scale datasets , 2017, Knowl. Based Syst..

[89]  Ibrahim Aljarah,et al.  A new clustering approach based on Glowworm Swarm Optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[90]  Hiroshi Motoda,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998, The Springer International Series in Engineering and Computer Science.

[91]  Fan Yu,et al.  The computing of the optimal power consumption for semi-track air-cushion vehicle using hybrid generalized extremal optimization , 2009 .

[92]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[93]  Edwin Lughofer,et al.  Performance comparison of generational and steady-state asynchronous multi-objective evolutionary algorithms for computationally-intensive problems , 2015, Knowl. Based Syst..