A Hybrid GA-GP Method for Feature Reduction in Classification

Feature reduction is an important pre-processing step in classification and other artificial intelligent applications. Its aim is to improve the quality of feature sets. There are two main types of feature reduction: feature construction and feature selection. Most current feature reduction algorithms focus on just one of the two types because they require different representations. This paper proposes a new representation which supports a feature reduction algorithm that combines feature selection and feature construction. The algorithm uses new genetic operators to update the new representation. The proposed algorithm is compared with two conventional feature selection algorithms, a genetic algorithms-based feature selection algorithm, and a genetic programming-based algorithm which evolves feature sets containing both original and high-level features. The experimental results on 10 different datasets show that the new representation can help to produce a smaller number of features and improve the classification accuracy over using all features on most datasets. In comparison with other feature selection or construction algorithms, the proposed algorithm achieves similar or better classification performance on all datasets.

[1]  Mengjie Zhang,et al.  A Filter Approach to Multiple Feature Construction for Symbolic Learning Classifiers Using Genetic Programming , 2012, IEEE Transactions on Evolutionary Computation.

[2]  Marc Parizeau,et al.  DEAP: evolutionary algorithms made easy , 2012, J. Mach. Learn. Res..

[3]  Bing Xue,et al.  Surrogate-Model Based Particle Swarm Optimisation with Local Search for Feature Selection in Classification , 2017, EvoApplications.

[4]  Gang Niu,et al.  Feature Selection Optimization , 2017 .

[5]  Asoke K. Nandi,et al.  Breast Cancer Diagnosis Using Genetic Programming Generated Feature , 2005 .

[6]  João Miguel da Costa Sousa,et al.  Modified binary PSO for feature selection using SVM applied to mortality prediction of septic patients , 2013, Appl. Soft Comput..

[7]  Mengjie Zhang,et al.  PSO and Statistical Clustering for Feature Selection: A New Representation , 2014, SEAL.

[8]  Clarimar José Coelho,et al.  Feature Selection using Genetic Algorithm: An Analysis of the Bias-Property for One-Point Crossover , 2016, GECCO.

[9]  S. C. Neoh,et al.  A Micro-GA Embedded PSO Feature Selection Approach to Intelligent Facial Emotion Recognition , 2017, IEEE Transactions on Cybernetics.

[10]  Mengjie Zhang,et al.  Genetic programming for feature construction and selection in classification on high-dimensional data , 2016, Memetic Comput..

[11]  Pramod Kumar Singh,et al.  Opposition chaotic fitness mutation based adaptive inertia weight BPSO for feature selection in text clustering , 2016, Appl. Soft Comput..

[12]  Andrew M. Tyrrell,et al.  Evolving Classifiers to Recognize the Movement Characteristics of Parkinson's Disease Patients , 2014, IEEE Transactions on Evolutionary Computation.

[13]  Chen-Chien James Hsu,et al.  Hybrid particle swarm optimization incorporating fuzzy reasoning and weighted particle , 2015, Neurocomputing.

[14]  Bing Xue,et al.  Mutual information for feature selection: estimation or counting? , 2016, Evol. Intell..

[15]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex adaptive systems.

[16]  David E. Goldberg,et al.  Genetic algorithms and Machine Learning , 1988, Machine Learning.

[17]  Claudio De Stefano,et al.  A GA-based feature selection approach with an application to handwritten character recognition , 2014, Pattern Recognit. Lett..

[18]  Xin Yao,et al.  A Survey on Evolutionary Computation Approaches to Feature Selection , 2016, IEEE Transactions on Evolutionary Computation.

[19]  Mengjie Zhang,et al.  Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms , 2014, Appl. Soft Comput..

[20]  Mengjie Zhang,et al.  Gaussian Transformation Based Representation in Particle Swarm Optimisation for Feature Selection , 2015, EvoApplications.