A binary ABC algorithm based on advanced similarity scheme for feature selection

Graphical abstractAn illustrative representation on how the features are selected. Display Omitted HighlightsA binary artificial bee colony (ABC) algorithm is proposed for feature selection.A comprehensive comparative study of the ABC PSO variants is presented.The superiority of the algorithm is demonstrated on both training and test sets.The times of appearance of each feature over 30 runs for each dataset are presented.The proposed algorithm performs better than the others. Feature selection is the basic pre-processing task of eliminating irrelevant or redundant features through investigating complicated interactions among features in a feature set. Due to its critical role in classification and computational time, it has attracted researchers' attention for the last five decades. However, it still remains a challenge. This paper proposes a binary artificial bee colony (ABC) algorithm for the feature selection problems, which is developed by integrating evolutionary based similarity search mechanisms into an existing binary ABC variant. The performance analysis of the proposed algorithm is demonstrated by comparing it with some well-known variants of the particle swarm optimization (PSO) and ABC algorithms, including standard binary PSO, new velocity based binary PSO, quantum inspired binary PSO, discrete ABC, modification rate based ABC, angle modulated ABC, and genetic algorithms on 10 benchmark datasets. The results show that the proposed algorithm can obtain higher classification performance in both training and test sets, and can eliminate irrelevant and redundant features more effectively than the other approaches. Note that all the algorithms used in this paper except for standard binary PSO and GA are employed for the first time in feature selection.

[1]  Emrah Hancer,et al.  AUTOMATIC CLUSTERING WITH GLOBAL BEST ARTIFICIAL BEE COLONY ALGORITHM , 2014 .

[2]  Mengjie Zhang,et al.  Binary PSO and Rough Set Theory for Feature Selection: a Multi-objective filter Based Approach , 2014, Int. J. Comput. Intell. Appl..

[3]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[4]  Ali Husseinzadeh Kashan,et al.  DisABC: A new artificial bee colony algorithm for binary optimization , 2012, Appl. Soft Comput..

[5]  Thomas G. Dietterich,et al.  Learning Boolean Concepts in the Presence of Many Irrelevant Features , 1994, Artif. Intell..

[6]  Nihat Yilmaz,et al.  Feature Selection Method Based on Artificial Bee Colony Algorithm and Support Vector Machines for Medical Datasets Classification , 2013, TheScientificWorldJournal.

[7]  Yew-Soon Ong,et al.  Memetic Computation—Past, Present & Future [Research Frontier] , 2010, IEEE Computational Intelligence Magazine.

[8]  Hossein Nezamabadi-pour,et al.  An advanced ACO algorithm for feature subset selection , 2015, Neurocomputing.

[9]  Rafael Ramírez,et al.  An evolutionary computation approach to cognitive states classification , 2007, 2007 IEEE Congress on Evolutionary Computation.

[10]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[11]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[12]  Qian Yin,et al.  Ant Colony Optimization Algorithm for Feature Selection and Classification of Multispectral Remote Sensing Image , 2008, IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium.

[13]  A. Wayne Whitney,et al.  A Direct Method of Nonparametric Measurement Selection , 1971, IEEE Transactions on Computers.

[14]  Hélio Pedrini,et al.  Data feature selection based on Artificial Bee Colony algorithm , 2013, EURASIP J. Image Video Process..

[15]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[16]  A. Sankar,et al.  Pattern Matching based Classification using Ant Colony Optimization based Feature Selection , 2015, Appl. Soft Comput..

[17]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[18]  Dervis Karaboga,et al.  Artificial bee colony algorithm , 2010, Scholarpedia.

[19]  Hao Dong,et al.  An improved particle swarm optimization for feature selection , 2011 .

[20]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[21]  Mengjie Zhang,et al.  New fitness functions in binary particle swarm optimisation for feature selection , 2012, 2012 IEEE Congress on Evolutionary Computation.

[22]  R. R. Rajalaxmi Artificial Bee Colony based Feature Selection for Effective Cardiovascular Disease Diagnosis , 2014 .

[23]  Abdolreza Hatamlou,et al.  Black hole: A new heuristic optimization approach for data clustering , 2013, Inf. Sci..

[24]  Dervis Karaboga,et al.  A comprehensive survey: artificial bee colony (ABC) algorithm and applications , 2012, Artificial Intelligence Review.

[25]  Anil K. Jain,et al.  Dimensionality reduction using genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[26]  P. Jaccard THE DISTRIBUTION OF THE FLORA IN THE ALPINE ZONE.1 , 1912 .

[27]  Nikhil R. Pal,et al.  Genetic programming for simultaneous feature selection and classifier design , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Dervis Karaboga,et al.  Improved clustering criterion for image clustering with artificial bee colony algorithm , 2014, Pattern Analysis and Applications.

[29]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[30]  Mengjie Zhang,et al.  Single Feature Ranking and Binary Particle Swarm Optimisation Based Feature Subset Ranking for Feature Selection , 2012, ACSC.

[31]  Sheng Ding,et al.  Feature Selection Based F-Score and ACO Algorithm in Support Vector Machine , 2009, 2009 Second International Symposium on Knowledge Acquisition and Modeling.

[32]  B. Bonev Feature Selection based on Information Theory , 2010 .

[33]  Dervis Karaboga,et al.  A modified Artificial Bee Colony algorithm for real-parameter optimization , 2012, Inf. Sci..

[34]  J. Anuradha,et al.  A Survey on Particle Swarm Optimization in Feature Selection , 2011 .

[35]  M. A. Khanesar,et al.  A novel binary particle swarm optimization , 2007, 2007 Mediterranean Conference on Control & Automation.

[36]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[37]  M. Akila,et al.  Performance of classification using a hybrid distance measure with artificial bee colony algorithm for feature selection in keystroke dynamics , 2013, Int. J. Comput. Intell. Stud..

[38]  Jihoon Yang,et al.  Feature Subset Selection Using a Genetic Algorithm , 1998, IEEE Intell. Syst..

[39]  Richard Jensen,et al.  Performing Feature Selection with ACO , 2006, Swarm Intelligence in Data Mining.

[40]  Mengjie Zhang,et al.  Particle Swarm Optimisation and Statistical Clustering for Feature Selection , 2013, Australasian Conference on Artificial Intelligence.

[41]  Carlos Cotta,et al.  Memetic algorithms and memetic computing optimization: A literature review , 2012, Swarm Evol. Comput..

[42]  S. N. Sivanandam,et al.  Genetic Algorithm Implementation Using Matlab , 2008 .

[43]  Ali Husseinzadeh Kashan,et al.  A novel differential evolution algorithm for binary optimization , 2012, Computational Optimization and Applications.

[44]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[45]  Jong-Bae Park,et al.  A New Quantum-Inspired Binary PSO: Application to Unit Commitment Problems for Power Systems , 2010, IEEE Transactions on Power Systems.

[46]  S. Kanmani,et al.  Feature selection optimization through enhanced Artificial Bee Colony algorithm , 2013, 2013 International Conference on Recent Trends in Information Technology (ICRTIT).

[47]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[48]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[49]  Nic James,et al.  Relationship between Repeated Sprint Ability and Aerobic Capacity in Professional Soccer Players , 2013, TheScientificWorldJournal.

[50]  C. Tappert,et al.  A Survey of Binary Similarity and Distance Measures , 2010 .

[51]  Mengjie Zhang,et al.  A Dimension Reduction Approach to Classification Based on Particle Swarm Optimisation and Rough Set Theory , 2012, Australasian Conference on Artificial Intelligence.

[52]  M. S. Kiran,et al.  XOR-based artificial bee colony algorithm for binary optimization , 2013 .

[53]  Mengjie Zhang,et al.  An archive based particle swarm optimisation for feature selection in classification , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[54]  Eibe Frank,et al.  Large-scale attribute selection using wrappers , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[55]  Seyed Mohammad Mirjalili,et al.  Ions motion algorithm for solving optimization problems , 2015, Appl. Soft Comput..

[56]  Esra Saraç,et al.  An Ant Colony Optimization Based Feature Selection for Web Page Classification , 2014, TheScientificWorldJournal.

[57]  Mengjie Zhang,et al.  Improving feature ranking for biomarker discovery in proteomics mass spectrometry data using genetic programming , 2014, Connect. Sci..

[58]  Zexuan Zhu,et al.  Wrapper–Filter Feature Selection Algorithm Using a Memetic Framework , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[59]  Nambiraj Suguna,et al.  An Independent Rough Set Approach Hybrid with Artificial Bee Colony Algorithm for Dimensionality Reduction , 2011 .

[60]  Beatriz de la Iglesia,et al.  Evolutionary computation for feature selection in classification problems , 2013, WIREs Data Mining Knowl. Discov..

[61]  He Ming A Rough Set Based Hybrid Method to Feature Selection , 2008, 2008 International Symposium on Knowledge Acquisition and Modeling.

[62]  Mengjie Zhang,et al.  Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach , 2013, IEEE Transactions on Cybernetics.

[63]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[64]  Swagatam Das,et al.  Synergizing fitness learning with proximity-based food source selection in artificial bee colony algorithm for numerical optimization , 2013, Appl. Soft Comput..

[65]  Andries Petrus Engelbrecht,et al.  Binary artificial bee colony optimization , 2011, 2011 IEEE Symposium on Swarm Intelligence.

[66]  Rich Caruana,et al.  Greedy Attribute Selection , 1994, ICML.

[67]  Andrew Lewis,et al.  S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization , 2013, Swarm Evol. Comput..

[68]  Dervis Karaboga,et al.  AN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION , 2005 .

[69]  Abdolreza Hatamlou,et al.  Heart: a novel optimization algorithm for cluster analysis , 2014, Progress in Artificial Intelligence.

[70]  Mengjie Zhang,et al.  A Multi-objective Feature Selection Approach Based on Binary PSO and Rough Set Theory , 2013, EvoCOP.

[71]  Laura Igual,et al.  Robust gait-based gender classification using depth cameras , 2013, EURASIP Journal on Image and Video Processing.

[72]  Nasser Ghasem-Aghaee,et al.  A novel ACO-GA hybrid algorithm for feature selection in protein function prediction , 2009, Expert Syst. Appl..

[73]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[74]  Alper Ekrem Murat,et al.  A discrete particle swarm optimization method for feature selection in binary classification problems , 2010, Eur. J. Oper. Res..

[75]  Dervis Karaboga,et al.  Color Image Quantization: A Short Review and an Application with Artificial Bee Colony Algorithm , 2014, Informatica.