Elitism-based multi-objective differential evolution with extreme learning machine for feature selection: a novel searching technique

ABSTRACT The features related to the real world data may be redundant and erroneous in nature. The vital role of feature selection (FS) in handling such type of features cannot be ignored in the area of computational learning. The two most commonly used objectives for FS are the maximisation of the accuracy and minimisation of the number of features. This paper presents an Elitism-based Multi-objective Differential Evolution algorithm for FS and the novelty lies in the searching process which uses Minkowski Score (MS) and simultaneously optimises three objectives. The MS is considered as the third objective to keep track of the feature subset which is capable enough to produce a good classification result even if the average accuracy is poor. Extreme Learning Machine because of its fast learning speed and high efficiency has been considered with this multi-objective approach as a classifier for FS. Twenty-one benchmark datasets have been considered for performance evaluation. Moreover, the selected feature subsets are tested using 10-fold cross-validation. A comparative analysis of the proposed approach with two classical models, three single objective algorithms, and four multi-objective algorithms has been carried out to test the efficacy of the model.

[1]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[2]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[3]  Mengjie Zhang,et al.  Multi-objective Feature Selection in Classification: A Differential Evolution Approach , 2014, SEAL.

[4]  Adel Al-Jumaily,et al.  Feature subset selection using differential evolution and a statistical repair mechanism , 2011, Expert Syst. Appl..

[5]  Xin-She Yang,et al.  BBA: A Binary Bat Algorithm for Feature Selection , 2012, 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images.

[6]  Naif Alajlan,et al.  Differential Evolution Extreme Learning Machine for the Classification of Hyperspectral Images , 2014, IEEE Geoscience and Remote Sensing Letters.

[7]  Swagatam Das,et al.  Automatic Clustering Using an Improved Differential Evolution Algorithm , 2007 .

[8]  Eibe Frank,et al.  Large-scale attribute selection using wrappers , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[9]  Xin Yao,et al.  A Survey on Evolutionary Computation Approaches to Feature Selection , 2016, IEEE Transactions on Evolutionary Computation.

[10]  Pradipta Kishore Dash,et al.  Data decomposition based fast reduced kernel extreme learning machine for currency exchange rate forecasting and trend analysis , 2018, Expert Syst. Appl..

[11]  Rich Caruana,et al.  Greedy Attribute Selection , 1994, ICML.

[12]  Rommel M. Barbosa,et al.  Geographical recognition of Syrah wines by combining feature selection with Extreme Learning Machine , 2018 .

[13]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[14]  Behzad Nazari,et al.  Effective pixel classification of Mars images based on ant colony optimization feature selection and extreme learning machine , 2017, Neurocomputing.

[15]  Dianhui Wang,et al.  Extreme learning machines: a survey , 2011, Int. J. Mach. Learn. Cybern..

[16]  Bishwajit Chakraborty,et al.  Genetic algorithm with fuzzy fitness function for feature selection , 2002, Industrial Electronics, 2002. ISIE 2002. Proceedings of the 2002 IEEE International Symposium on.

[17]  Rami N. Khushaba,et al.  Feature subset selection using differential evolution and a wheel based search strategy , 2013, Swarm Evol. Comput..

[18]  A. Wayne Whitney,et al.  A Direct Method of Nonparametric Measurement Selection , 1971, IEEE Transactions on Computers.

[19]  Banshidhar Majhi,et al.  Discrete ripplet-II transform and modified PSO based improved evolutionary extreme learning machine for pathological brain detection , 2017, Neurocomputing.

[20]  Claire Cardie,et al.  Using Decision Trees to Improve Case-Based Learning , 1993, ICML.

[21]  Saeid Nahavandi,et al.  Extreme learning machine based transfer learning algorithms: A survey , 2017, Neurocomputing.

[22]  Jason Jianjun Gu,et al.  An Efficient Method for Traffic Sign Recognition Based on Extreme Learning Machine , 2017, IEEE Transactions on Cybernetics.

[23]  Marco Laumanns,et al.  Performance assessment of multiobjective optimizers: an analysis and review , 2003, IEEE Trans. Evol. Comput..

[24]  Johannes Bader,et al.  Hypervolume-based search for multiobjective optimization: Theory and methods , 2010 .

[25]  Nikhil R. Pal,et al.  Genetic programming for simultaneous feature selection and classifier design , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[27]  Ana Carolina Lorena,et al.  Use of Multiobjective Genetic Algorithms in Feature Selection , 2010, 2010 Eleventh Brazilian Symposium on Neural Networks.

[28]  Fakhri Karray,et al.  Multi-objective Feature Selection with NSGA II , 2007, ICANNGA.

[29]  Swagatam Das,et al.  Simultaneous feature selection and weighting - An evolutionary multi-objective optimization approach , 2015, Pattern Recognit. Lett..

[30]  Pravat Kumar Rout,et al.  Elitism based Multi-Objective Differential Evolution for feature selection: A filter approach with an efficient redundancy measure , 2017, J. King Saud Univ. Comput. Inf. Sci..

[31]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[32]  Ujjwal Maulik,et al.  A Survey of Multiobjective Evolutionary Algorithms for Data Mining: Part I , 2014, IEEE Transactions on Evolutionary Computation.

[33]  Adel Al-Jumaily,et al.  A Combined Ant Colony and Differential Evolution Feature Selection Algorithm , 2008, ANTS Conference.

[34]  Anne Auger,et al.  Theory of the hypervolume indicator: optimal μ-distributions and the choice of the reference point , 2009, FOGA '09.

[35]  Zhigang Shang,et al.  Feature Selection based on manifold-learning with dynamic constraint handling differential evolution , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[36]  Luis Magdalena,et al.  A Multiobjective Genetic Learning Process for joint Feature Selection and Granularity and Contexts Learning in Fuzzy Rule-Based Classification Systems , 2003 .

[37]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[38]  Mengjie Zhang,et al.  Particle Swarm Optimization for Feature Selection in Classification: A Multi-Objective Approach , 2013, IEEE Transactions on Cybernetics.

[39]  Panos M. Pardalos,et al.  Feature selection based on meta-heuristics for biomedicine , 2014, Optim. Methods Softw..

[40]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[41]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[42]  Larry A. Rendell,et al.  A Practical Approach to Feature Selection , 1992, ML.

[43]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[44]  B. Chakraborty Feature subset selection by particle swarm optimization with fuzzy fitness function , 2008, 2008 3rd International Conference on Intelligent System and Knowledge Engineering.

[45]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..