A multi-objective evolutionary algorithm-based ensemble optimizer for feature selection and classification with neural network models

In this paper, we propose a new multi-objective evolutionary algorithm-based ensemble optimizer coupled with neural network models for undertaking feature selection and classification problems. Specifically, the Modified micro Genetic Algorithm (MmGA) is used to form the ensemble optimizer. The aim of the MmGA-based ensemble optimizer is two-fold, i.e. to select a small number of input features for classification and to improve the classification performances of neural network models. To evaluate the effectiveness of the proposed system, a number of benchmark problems are first used, and the results are compared with those from other methods. The applicability of the proposed system to a human motion detection and classification task is then evaluated. The outcome positively demonstrates that the proposed MmGA-based ensemble optimizer is able to improve the classification performances of neural network models with a smaller number of input features.

[1]  David Howard,et al.  A Comparison of Feature Extraction Methods for the Classification of Dynamic Activities From Accelerometer Data , 2009, IEEE Transactions on Biomedical Engineering.

[2]  Johannes Peltola,et al.  Activity classification using realistic data from wearable sensors , 2006, IEEE Transactions on Information Technology in Biomedicine.

[3]  B. Efron.  Nonparametric standard errors and confidence intervals , 1981 .

[4]  Mario Marchand,et al.  Feature Selection with Conjunctions of Decision Stumps and Learning from Microarray Data , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Kay Chen Tan,et al.  Hybrid Multiobjective Evolutionary Design for Artificial Neural Networks , 2008, IEEE Transactions on Neural Networks.

[6]  Mykola Pechenizkiy,et al.  Diversity in search strategies for ensemble feature selection , 2005, Inf. Fusion.

[7]  DebK.,et al.  A fast and elitist multiobjective genetic algorithm , 2002 .

[8]  H WittenIan,et al.  The WEKA data mining software , 2009 .

[9]  Luiz Eduardo Soares de Oliveira,et al.  Feature selection for ensembles applied to handwriting recognition , 2006, International Journal of Document Analysis and Recognition (IJDAR).

[10]  Hitoshi Iba,et al.  Classification of Gene Expression Data by Majority Voting Genetic Programming Classifier , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[11]  Ching-Lai Hwang,et al.  Fuzzy Multiple Attribute Decision Making - Methods and Applications , 1992, Lecture Notes in Economics and Mathematical Systems.

[12]  Juan José Rodríguez Diez,et al.  Random Subspace Ensembles for fMRI Classification , 2010, IEEE Transactions on Medical Imaging.

[13]  Ludmila I. Kuncheva,et al.  Switching between selection and fusion in combining classifiers: an experiment , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[14]  B. Efron,et al.  The Jackknife: The Bootstrap and Other Resampling Plans. , 1983 .

[15]  Kenneth J. Mackin,et al.  Ensemble of artificial neural network based land cover classifiers using satellite data , 2007, 2007 IEEE International Conference on Systems, Man and Cybernetics.

[16]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[17]  Hui-Ling Huang,et al.  ESVM: Evolutionary support vector machine for automatic feature selection and classification of microarray data , 2007, Biosyst..

[18]  Harris Drucker,et al.  Boosting and Other Ensemble Methods , 1994, Neural Computation.

[19]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[20]  Silvio Savarese,et al.  Cross-view action recognition via view knowledge transfer , 2011, CVPR 2011.

[21]  A Coello CoelloC..  Evolutionary multi-objective optimization , 2006 .

[22]  Hitoshi Iba,et al.  Prediction of Cancer Class with Majority Voting Genetic Programming Classifier Using Gene Expression Data , 2009, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[23]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.

[24]  Ben Taskar,et al.  Parsing human motion with stretchable models , 2011, CVPR 2011.

[25]  Anthony Rowe,et al.  Location and Activity Recognition Using eWatch: A Wearable Sensor Platform , 2006, Ambient Intelligence in Everyday.

[26]  C. Hwang.  Multiple Objective Decision Making - Methods and Applications: A State-of-the-Art Survey , 1979 .

[27]  Teresa Bernarda Ludermir,et al.  Hybrid Training Method for MLP: Optimization of Architecture and Training , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[29]  T. Fearn.  The Jackknife , 2000 .

[30]  Anil K. Jain,et al.  Dimensionality reduction using genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[31]  Gonzalo Mart.  Switching Class Labels to Generate Classication Ensembles , 2005 .

[32]  Jean-Claude Laprie,et al.  Trend Analysis , 1996, The SAGE Encyclopedia of Research Design.

[33]  C. A. Coello Coello,et al.  Multiobjective structural optimization using a microgenetic algorithm , 2005 .

[34]  Byung Ro Moon,et al.  Hybrid Genetic Algorithms for Feature Selection , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Jiawei Han,et al.  Knowledge transfer via multiple model local structure mapping , 2008, KDD.

[36]  A. Redish,et al.  Addiction as a Computational Process Gone Awry , 2004, Science.

[37]  Ying Wu,et al.  Mining actionlet ensemble for action recognition with depth cameras , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Antonio J. Nebro,et al.  jMetal: A Java framework for multi-objective optimization , 2011, Adv. Eng. Softw..

[39]  Daniel Hernández-Lobato,et al.  Class-switching neural network ensembles , 2008, Neurocomputing.

[40]  Kazuyuki Murase,et al.  Ensembles of Neural Networks Based on the Alteration of Input Feature Values , 2012, Int. J. Neural Syst..

[41]  Jun Yang,et al.  Physical Activity Recognition with Mobile Phones: Challenges, Methods, and Applications , 2010 .

[42]  Lin Sun,et al.  Activity Recognition on an Accelerometer Embedded Mobile Phone with Varying Positions and Orientations , 2010, UIC.

[43]  Ashfaqur Rahman,et al.  Novel Layered Clustering-Based Approach for Generating Ensemble of Classifiers , 2011, IEEE Transactions on Neural Networks.

[44]  Roger Ratcliff,et al.  Reinforcement-Based Decision Making in Corticostriatal Circuits: Mutual Constraints by Neurocomputational and Diffusion Models , 2012, Neural Computation.

[45]  Loris Nanni,et al.  A simple method for improving local binary patterns by considering non-uniform patterns , 2012, Pattern Recognit..

[46]  Marco Dorigo,et al.  Message-Based Bucket Brigade: An Algorithm for the Apportionment of Credit Problem , 1991, EWSL.

[47]  Huan Liu.  Feature Selection , 2010, Encyclopedia of Machine Learning.

[48]  Chee Peng Lim,et al.  A Modified micro Genetic Algorithm for undertaking Multi-Objective Optimization Problems , 2013, J. Intell. Fuzzy Syst..

[49]  J. Kiefer,et al.  The interplay of optimality and combinatorics in experimental design , 1981 .

[50]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[51]  Lior Rokach,et al.  Ensemble Methods in Supervised Learning , 2010, Data Mining and Knowledge Discovery Handbook.

[52]  David W. Corne,et al.  Approximating the Nondominated Front Using the Pareto Archived Evolution Strategy , 2000, Evolutionary Computation.

[53]  Anne M. P. Canuto,et al.  Optimization techniques for the selection of members and attributes in ensemble systems , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[54]  F. Cheng,et al.  GENERALIZED CENTER METHOD FOR MULTIOBJECTIVE ENGINEERING OPTIMIZATION , 1999 .

[55]  Edmund K. Burke,et al.  USING DIVERSITY TO GUIDE THE SEARCH IN MULTI-OBJECTIVE OPTIMIZATION , 2004 .

[56]  Anany Levitin,et al.  Introduction to the Design and Analysis of Algorithms , 2002 .

[57]  Juan José Rodríguez Diez,et al.  Rotation Forest: A New Classifier Ensemble Method , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[58]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  Takashi Yamaguchi,et al.  Artificial neural network ensemble-based land-cover classifiers using MODIS data , 2008, Artificial Life and Robotics.

[60]  Carlos A. Coello Coello,et al.  Evolutionary multi-objective optimization: a historical view of the field , 2006, IEEE Comput. Intell. Mag..

[61]  Adrian E. Raftery,et al.  Weather Forecasting with Ensemble Methods , 2005, Science.

[62]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[63]  G.A.L. Meijer,et al.  Methods to assess physical activity with special reference to motion sensors and accelerometers , 1991, IEEE Transactions on Biomedical Engineering.

[64]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[65]  Kay Soon Low,et al.  A Factorized Quaternion Approach to Determine the Arm Motions Using Triaxial Accelerometers With Anatomical and Sensor Constraints , 2012, IEEE Transactions on Instrumentation and Measurement.

[66]  Herbert A. Simon,et al.  Applications of machine learning and rule induction , 1995, CACM.

[67]  Giorgio Valentini,et al.  Bio-molecular cancer prediction with random subspace ensembles of support vector machines , 2005, Neurocomputing.

[68]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[69]  Lakhmi C. Jain,et al.  Designing classifier fusion systems by genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[70]  Xin Yao,et al.  Ensemble learning via negative correlation , 1999, Neural Networks.

[71]  MengChu Zhou,et al.  Body Sensors Applied in Pacemakers: A Survey , 2012, IEEE Sensors Journal.

[72]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[73]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .