Recent Developments on Evolutionary Computation Techniques to Feature Construction

The quality of the search space is an important factor that influences the performance of any machine learning algorithm including its classification. The attributes that define the search space can be poorly understood or inadequate, thereby making it difficult to discover high quality knowledge and understanding. Feature construction (FC) and feature selection (FS) are two pre-processing steps that can be used to improve the feature space quality, by enhancing the classifier performance in terms of accuracy, complexity, speed and interpretability. While FS aims to choose a set of informative features for improving the performance, FC can enhance the classification performance by evolving new features out of the original ones. The evolved features are expected to have more predictive value than the originals that make them up. Over the past few decades, several evolutionary computation (EC) methods have been proposed in the area of FC. This paper gives an overview of the literature on EC for FC. Here, we focus mainly on filter, wrapper and embedded methods, in which the contributions of these different methods are identified. Furthermore, some open challenges and current issues are also discussed in order to identify promising areas for future research.

[1]  Mengjie Zhang,et al.  PSO for feature construction and binary classification , 2013, GECCO '13.

[2]  Dah-Jye Lee,et al.  A feature construction method for general object recognition , 2013, Pattern Recognit..

[3]  Erik D. Goodman,et al.  On Prediction of Epileptic Seizures by Computing Multiple Genetic Programming Artificial Features , 2005, EuroGP.

[4]  Cardona Alzate,et al.  Predicción y selección de variables con bosques aleatorios en presencia de variables correlacionadas , 2020 .

[5]  Qingzhong Liu,et al.  Distance Metric Learning and Support Vector Machines for Classification of Mass Spectrometry Proteomics Data , 2008, 2008 Seventh International Conference on Machine Learning and Applications.

[6]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[7]  Ian Welch,et al.  An efficient approach for feature construction of high-dimensional microarray data by random projections , 2018, PloS one.

[8]  George D. Smith,et al.  Evolutionary constructive induction , 2005, IEEE Transactions on Knowledge and Data Engineering.

[9]  Esmaeil Hadavandi,et al.  MBCGP-FE: A modified balanced cartesian genetic programming feature extractor , 2017, Knowl. Based Syst..

[10]  George D. Smith,et al.  Evolutionary Feature Construction Using Information Gain and Gini Index , 2004, EuroGP.

[11]  Asoke K. Nandi,et al.  Breast Cancer Diagnosis Using Genetic Programming Generated Feature , 2005, 2005 IEEE Workshop on Machine Learning for Signal Processing.

[12]  Mengjie Zhang,et al.  Multiple feature construction in classification on high-dimensional data using GP , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[13]  Ron Kohavi,et al.  Irrelevant Features and the Subset Selection Problem , 1994, ICML.

[14]  Mengjie Zhang,et al.  Multiple feature construction for effective biomarker identification and classification using genetic programming , 2014, GECCO.

[15]  David García,et al.  A two-step approach of feature construction for a genetic learning algorithm , 2011, 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011).

[16]  Mengjie Zhang,et al.  Genetic Programming with Embedded Feature Construction for High-Dimensional Symbolic Regression , 2017 .

[17]  Bing Xue,et al.  A Hybrid GA-GP Method for Feature Reduction in Classification , 2017, SEAL.

[18]  A. E. Eiben,et al.  From evolutionary computation to the evolution of things , 2015, Nature.

[19]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection: A Data Mining Perspective , 1998 .

[20]  Alex Alves Freitas,et al.  Constructing X-of-n Attributes With A Genetic Algorithm , 2002, GECCO.

[21]  Halina Kwasnicka,et al.  Feature Set Reduction by Evolutionary Selection and Construction , 2010, KES-AMSTA.

[22]  Mengjie Zhang,et al.  Genetic programming based feature construction for classification with incomplete data , 2017, GECCO.

[23]  Hossein Nezamabadi-pour,et al.  Improved PSO-based feature construction algorithm using Feature Selection Methods , 2017, 2017 2nd Conference on Swarm Intelligence and Evolutionary Computation (CSIEC).

[24]  M. El-Sharkawi,et al.  Introduction to Evolutionary Computation , 2008 .

[25]  Mengjie Zhang,et al.  A new GP-based wrapper feature construction approach to classification and biomarker identification , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[26]  Masanori Suganuma,et al.  Hierarchical feature construction for image classification using Genetic Programming , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[27]  Mengjie Zhang,et al.  Feature Construction Using Genetic Programming for Figure-Ground Image Segmentation , 2017 .

[28]  Mengjie Zhang,et al.  New Representations in PSO for Feature Construction in Classification , 2014, EvoApplications.

[29]  David García,et al.  A feature construction approach for genetic iterative rule learning algorithm , 2014, J. Comput. Syst. Sci..

[30]  Krzysztof J. Cios,et al.  Multi-objective genetic programming for feature extraction and data visualization , 2015, Soft Computing.

[31]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[32]  Mengjie Zhang,et al.  A Multi-objective Genetic Programming Biomarker Detection Approach in Mass Spectrometry Data , 2016, EvoApplications.

[33]  Mengjie Zhang,et al.  Genetic programming for feature construction and selection in classification on high-dimensional data , 2016, Memetic Comput..

[34]  Kenneth DeJong,et al.  Genetic algorithms as a tool for restructuring feature space representations , 1995, Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence.

[35]  Javier Del Ser,et al.  Hybridizing Cartesian Genetic Programming and Harmony Search for adaptive feature construction in supervised learning problems , 2017, Appl. Soft Comput..

[36]  Xin Yao,et al.  A Survey on Evolutionary Computation Approaches to Feature Selection , 2016, IEEE Transactions on Evolutionary Computation.

[37]  Rayner Alfred,et al.  DARA: Data Summarisation with Feature Construction , 2008, 2008 Second Asia International Conference on Modelling & Simulation (AMS).

[38]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection , 1998 .

[39]  Mengjie Zhang,et al.  Evolutionary computation for feature manipulation: Key challenges and future directions , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[40]  Eduardo Pérez,et al.  Evolutionary multi-feature construction for data reduction: A case study , 2009, Appl. Soft Comput..

[41]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[42]  Mengjie Zhang,et al.  Using Feature Clustering for GP-Based Feature Construction on High-Dimensional Data , 2017, EuroGP.

[43]  Mengjie Zhang,et al.  A Filter Approach to Multiple Feature Construction for Symbolic Learning Classifiers Using Genetic Programming , 2012, IEEE Transactions on Evolutionary Computation.

[44]  Krzysztof Krawiec,et al.  Genetic Programming-based Construction of Features for Machine Learning and Knowledge Discovery Tasks , 2002, Genetic Programming and Evolvable Machines.