An efficient binary Salp Swarm Algorithm with crossover scheme for feature selection problems

Abstract Searching for the (near) optimal subset of features is a challenging problem in the process of feature selection (FS). In the literature, Swarm Intelligence (SI) algorithms show superior performance in solving this problem. This motivated our attempts to test the performance of the newly proposed Salp Swarm Algorithm (SSA) in this area. As such, two new wrapper FS approaches that use SSA as the search strategy are proposed. In the first approach, eight transfer functions are employed to convert the continuous version of SSA to binary. In the second approach, the crossover operator is used in addition to the transfer functions to replace the average operator and enhance the exploratory behavior of the algorithm. The proposed approaches are benchmarked on 22 well-known UCI datasets and the results are compared with 5 FS methods: Binary Grey Wolf Optimizer (BGWO), Binary Gravitational Search Algorithms (BGSA), Binary Bat Algorithm (BBA), Binary Particle Swarm Optimization (BPSO), and Genetic Algorithm (GA). The paper also considers an extensive study of the parameter setting for the proposed technique. From the results, it is observed that the proposed approach significantly outperforms others on around 90% of the datasets.

[1]  S. Kanmani,et al.  A hybrid algorithm using ant and bee colony optimization for feature selection and classification (AC-ABC Hybrid) , 2017, Swarm Evol. Comput..

[2]  Andrew Lewis,et al.  The Whale Optimization Algorithm , 2016, Adv. Eng. Softw..

[3]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[4]  Seyedali Mirjalili,et al.  Dragonfly algorithm: a new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems , 2015, Neural Computing and Applications.

[5]  Janez Brest,et al.  Multi-Objective Differential Evolution for feature selection in Facial Expression Recognition systems , 2017, Expert Syst. Appl..

[6]  Kok-Leong Ong,et al.  Feature selection for high dimensional imbalanced class data using harmony search , 2017, Eng. Appl. Artif. Intell..

[7]  Aboul Ella Hassanien,et al.  Binary ant lion approaches for feature selection , 2016, Neurocomputing.

[8]  Pramod Kumar Singh,et al.  Opposition chaotic fitness mutation based adaptive inertia weight BPSO for feature selection in text clustering , 2016, Appl. Soft Comput..

[9]  A. Rezaee Jordehi,et al.  An efficient chaotic water cycle algorithm for optimization tasks , 2015, Neural Computing and Applications.

[10]  Lei Lu,et al.  Dynamic Genetic Algorithm-based Feature Selection Scheme for Machine Health Prognostics , 2016 .

[11]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[12]  Sunanda Das,et al.  Ensemble feature selection using bi-objective genetic algorithm , 2017, Knowl. Based Syst..

[13]  Rahim Ali Abbaspour,et al.  Enhanced Chaotic Grey Wolf Optimizer for Real-World Optimization Problems: A Comparative Study , 2018 .

[14]  Razieh Sheikhpour,et al.  Particle swarm optimization for bandwidth determination and feature selection of kernel density estimation based classifiers in diagnosis of breast cancer , 2016, Appl. Soft Comput..

[15]  Hossein Nezamabadi-pour,et al.  Facing the classification of binary problems with a GSA-SVM hybrid system , 2013, Math. Comput. Model..

[16]  S. Gunasundari,et al.  Velocity Bounded Boolean Particle Swarm Optimization for improved feature selection in liver and kidney disease diagnosis , 2016, Expert Syst. Appl..

[17]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[18]  Hossam Faris,et al.  A multi-verse optimizer approach for feature selection and optimizing SVM parameters based on a robust system architecture , 2017, Neural Computing and Applications.

[19]  Hossein Nezamabadi-pour,et al.  BGSA: binary gravitational search algorithm , 2010, Natural Computing.

[20]  Franz Pernkopf,et al.  Bayesian network classifiers versus selective k-NN classifier , 2005, Pattern Recognit..

[21]  Hossam Faris,et al.  Grey wolf optimizer: a review of recent variants and applications , 2017, Neural Computing and Applications.

[22]  Nizamettin Aydin,et al.  Binary black hole algorithm for feature selection and classification on biological data , 2017, Appl. Soft Comput..

[23]  Majdi M. Mafarja,et al.  Hybrid Whale Optimization Algorithm with simulated annealing for feature selection , 2017, Neurocomputing.

[24]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[25]  Hossein Nezamabadi-pour,et al.  Feature subset selection using improved binary gravitational search algorithm , 2014, J. Intell. Fuzzy Syst..

[26]  Aboul Ella Hassanien,et al.  Binary grey wolf optimization approaches for feature selection , 2016, Neurocomputing.

[27]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[28]  Laith Mohammad Abualigah,et al.  A new feature selection method to improve the document clustering using particle swarm optimization algorithm , 2017, J. Comput. Sci..

[29]  Zhiwei Ye,et al.  A feature selection method based on modified binary coded ant colony optimization algorithm , 2016, Appl. Soft Comput..

[30]  Zarita Zainuddin,et al.  An enhanced harmony search based algorithm for feature selection: Applications in epileptic seizure detection and prediction , 2016, Comput. Electr. Eng..

[31]  Xin-She Yang,et al.  A wrapper approach for feature selection based on Bat Algorithm and Optimum-Path Forest , 2014, Expert Syst. Appl..

[32]  A. Rezaee Jordehi,et al.  Gaussian bare-bones water cycle algorithm for optimal reactive power dispatch in electrical power systems , 2017, Appl. Soft Comput..

[33]  Amir Hossein Alavi,et al.  Krill herd: A new bio-inspired optimization algorithm , 2012 .

[34]  Li Zhang,et al.  Differential evolution algorithm as a tool for optimal feature subset selection in motor imagery EEG , 2017, Expert Syst. Appl..

[35]  Selma Ayse Özel,et al.  A hybrid approach of differential evolution and artificial bee colony for feature selection , 2016, Expert Syst. Appl..

[36]  João Paulo Papa,et al.  A novel algorithm for feature selection using Harmony Search and its application for non-technical losses detection , 2011, Comput. Electr. Eng..

[37]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[38]  Clarence W. de Silva,et al.  Feature selection for ECG signal processing using improved genetic algorithm and empirical mode decomposition , 2016 .

[39]  Hossam Faris,et al.  Optimizing connection weights in neural networks using the whale optimization algorithm , 2016, Soft Computing.

[40]  Xin-She Yang,et al.  Binary bat algorithm , 2013, Neural Computing and Applications.

[41]  Hossein Nezamabadi-pour,et al.  An advanced ACO algorithm for feature subset selection , 2015, Neurocomputing.

[42]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[43]  Hossam Faris,et al.  Training radial basis function networks using biogeography-based optimizer , 2018, Neural Computing and Applications.

[44]  Xin-She Yang,et al.  Binary Flower Pollination Algorithm and Its Application to Feature Selection , 2015, Recent Advances in Swarm Intelligence and Evolutionary Computation.

[45]  Parham Moradi,et al.  A hybrid particle swarm optimization for feature subset selection by integrating a novel local search strategy , 2016, Appl. Soft Comput..

[46]  Crina Grosan,et al.  Experienced Gray Wolf Optimization Through Reinforcement Learning and Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[48]  Seyed Mohammad Mirjalili,et al.  The Ant Lion Optimizer , 2015, Adv. Eng. Softw..

[49]  Seyed Mohammad Mirjalili,et al.  Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm , 2015, Knowl. Based Syst..

[50]  Parham Pahlavani,et al.  An efficient modified grey wolf optimizer with Lévy flight for optimization tasks , 2017, Appl. Soft Comput..

[51]  Hossam Faris,et al.  Evolutionary Population Dynamics and Grasshopper Optimization approaches for feature selection problems , 2017, Knowl. Based Syst..

[52]  Wen-Tsao Pan,et al.  A new Fruit Fly Optimization Algorithm: Taking the financial distress model as an example , 2012, Knowl. Based Syst..

[53]  Uffe Kock Wiil,et al.  Weighted bee colony algorithm for discrete optimization problems with application to feature selection , 2015, Eng. Appl. Artif. Intell..

[54]  Hiroshi Motoda,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998, The Springer International Series in Engineering and Computer Science.

[55]  Hossam Faris,et al.  Salp Swarm Algorithm: A bio-inspired optimizer for engineering design problems , 2017, Adv. Eng. Softw..

[56]  Padhraic Smyth,et al.  From Data Mining to Knowledge Discovery in Databases , 1996, AI Mag..

[57]  Andrew Lewis,et al.  S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization , 2013, Swarm Evol. Comput..

[58]  Lichao Cao,et al.  Improved particle swarm optimization algorithm and its application in text feature selection , 2015, Appl. Soft Comput..

[59]  Hossam Faris,et al.  Simultaneous Feature Selection and Support Vector Machine Optimization Using the Grasshopper Optimization Algorithm , 2018, Cognitive Computation.

[60]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[61]  Mohammad Saniee Abadeh,et al.  Image steganalysis using a bee colony based feature selection algorithm , 2014, Eng. Appl. Artif. Intell..

[62]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.