Binary Giza Pyramids Construction For Feature Selection

Abstract Feature selection (FS) is considered a challenging machine learning problem that handles the large size of features. The main purpose of FS is to remove irrelevant and redundant variables to improve the performance of the learning algorithms. Consequently, the FS process is considered as an optimization problem where metaheuristics approaches prove efficiency in solving it. In this paper, we propose new binary versions of a new ancient inspired metaheuristic approach called Giza Pyramids Construction (GPC) to select the most relevant subset of features. The proposed binary versions of the algorithm called BGPC-S and BGPC-V are implemented with two transfer functions, with the main objective of maximizing classification accuracy and minimizing the number of selected features. The two versions of BGPC were compared to six well-known binary metaheuristics for feature selection problem, namely Binary Atom Search Optimisation (BASO), Binary Bat Algorithm (BBA), Binary Differential Evolution (BDE), Binary Grey Wolf Optimizer (BGWO), Binary Particle Swarm Optimization (BPSO), and Binary Harris Hawks Optimizer (BHHO), and evaluated over 20 datasets from the UCI repository. Experiments have demonstrated that the proposed approaches outperformed the other algorithms in terms of classification accuracy and the number of selected features.

[1]  Nasser Ghasem-Aghaee,et al.  Text feature selection using ant colony optimization , 2009, Expert Syst. Appl..

[2]  Mohamed Elhoseny,et al.  Feature selection based on artificial bee colony and gradient boosting decision tree , 2019, Appl. Soft Comput..

[3]  Huan Zhang,et al.  A Novel Feature Selection Method Based on Extreme Learning Machine and Fractional-Order Darwinian PSO , 2018, Comput. Intell. Neurosci..

[4]  Jianhua Zhang,et al.  An Accurate Very Short-Term Electric Load Forecasting Model with Binary Genetic Algorithm Based Feature Selection for Microgrid Applications , 2018, Electric Power Components and Systems.

[5]  M. M. A. Salama,et al.  Particle swarm optimization feature selection for the classification of conducting particles in transformer oil , 2011, IEEE Transactions on Dielectrics and Electrical Insulation.

[6]  Xin-She Yang,et al.  BBA: A Binary Bat Algorithm for Feature Selection , 2012, 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images.

[7]  Tadayoshi Fushiki,et al.  Estimation of prediction error by using K-fold cross-validation , 2011, Stat. Comput..

[8]  Jacob Zahavi,et al.  Using simulated annealing to optimize the feature selection problem in marketing applications , 2006, Eur. J. Oper. Res..

[9]  Aboul Ella Hassanien,et al.  Binary ant lion approaches for feature selection , 2016, Neurocomputing.

[10]  Jason Weston,et al.  Embedded Methods , 2006, Feature Extraction.

[11]  Deron Liang,et al.  Novel feature selection methods to financial distress prediction , 2014, Expert Syst. Appl..

[12]  Adel Al-Jumaily,et al.  Feature subset selection using differential evolution and a statistical repair mechanism , 2011, Expert Syst. Appl..

[13]  Jerzy W. Bala,et al.  Using Learning to Facilitate the Evolution of Features for Recognizing Visual Concepts , 1996, Evolutionary Computation.

[14]  Yuhui Shi,et al.  Metaheuristic research: a comprehensive survey , 2018, Artificial Intelligence Review.

[15]  Michel Toulouse,et al.  A multilevel tabu search algorithm for the feature selection problem in biomedical data , 2008, Comput. Math. Appl..

[16]  Li-Yeh Chuang,et al.  Improved binary PSO for feature selection using gene expression data , 2008, Comput. Biol. Chem..

[17]  Jin Song Dong,et al.  Binary Harris Hawks Optimizer for High-Dimensional, Low Sample Size Feature Selection , 2019, Algorithms for Intelligent Systems.

[18]  Mengjie Zhang,et al.  Differential evolution for filter feature selection based on information theory and feature ranking , 2018, Knowl. Based Syst..

[19]  Luiz Eduardo Soares de Oliveira,et al.  A Methodology for Feature Selection Using Multiobjective Genetic Algorithms for Handwritten Digit String Recognition , 2003, Int. J. Pattern Recognit. Artif. Intell..

[20]  Abdul Rahim Abdullah,et al.  Binary atom search optimisation approaches for feature selection , 2020, Connect. Sci..

[21]  Duoqian Miao,et al.  A rough set approach to feature selection based on ant colony optimization , 2010, Pattern Recognit. Lett..

[22]  Xiangyang Wang,et al.  Feature selection based on rough sets and particle swarm optimization , 2007, Pattern Recognit. Lett..

[23]  Andrew Lewis,et al.  S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization , 2013, Swarm Evol. Comput..

[24]  Hélio Pedrini,et al.  Data feature selection based on Artificial Bee Colony algorithm , 2013, EURASIP J. Image Video Process..

[25]  Sadoullah Ebrahimnejad,et al.  Giza Pyramids Construction: an ancient-inspired metaheuristic algorithm for optimization , 2020, Evol. Intell..

[26]  Aboul Ella Hassanien,et al.  Binary grey wolf optimization approaches for feature selection , 2016, Neurocomputing.

[27]  Oliver Kramer,et al.  K-Nearest Neighbors , 2013 .

[28]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[29]  Gang Chen,et al.  A novel wrapper method for feature selection and its applications , 2015, Neurocomputing.

[30]  Hongbin Zhang,et al.  Feature selection using tabu search method , 2002, Pattern Recognit..

[31]  E. Talbi,et al.  A Genetic Algorithm for Feature Selection in Data-Mining for Genetics , 2001 .

[32]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[33]  Amparo Alonso-Betanzos,et al.  Filter Methods for Feature Selection - A Comparative Study , 2007, IDEAL.

[34]  Jack Sklansky,et al.  A note on genetic algorithms for large-scale feature selection , 1989, Pattern Recognition Letters.