A Genetic Programming Approach to Hyper-Heuristic Feature Selection

Feature selection is the task of finding a subset of original features which is as small as possible yet still sufficiently describes the target concepts. Feature selection has been approached through both heuristic and meta-heuristic approaches. Hyper-heuristics are search methods for choosing or generating heuristics or components of heuristics, to solve a range of optimisation problems. This paper proposes a genetic-programming-based hyper-heuristic approach to feature selection. The proposed method evolves new heuristics using some basic components (building blocks). The evolved heuristics act as new search algorithms that can search the space of subsets of features. The classification performance (accuracy) of classifiers are improved by using small subsets of features found by evolved heuristics.

[1]  Christian Blum,et al.  Metaheuristics in combinatorial optimization: Overview and conceptual comparison , 2003, CSUR.

[2]  Mengjie Zhang,et al.  Genetic Programming for Feature Subset Ranking in Binary Classification Problems , 2009, EuroGP.

[3]  Alex S. Fukunaga,et al.  Automated Discovery of Local Search Heuristics for Satisfiability Testing , 2008, Evolutionary Computation.

[4]  Mengjie Zhang,et al.  Genetic Programming for Feature Ranking in Classification Problems , 2008, SEAL.

[5]  Graham Kendall,et al.  A Classification of Hyper-heuristic Approaches , 2010 .

[6]  Larry A. Rendell,et al.  The Feature Selection Problem: Traditional Methods and a New Algorithm , 1992, AAAI.

[7]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[8]  Larry Bull,et al.  Genetic Programming with a Genetic Algorithm for Feature Construction and Selection , 2005, Genetic Programming and Evolvable Machines.

[9]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[10]  Mengjie Zhang,et al.  Single Feature Ranking and Binary Particle Swarm Optimisation Based Feature Subset Ranking for Feature Selection , 2012, ACSC.

[11]  A. Wayne Whitney,et al.  A Direct Method of Nonparametric Measurement Selection , 1971, IEEE Transactions on Computers.

[12]  Kenneth A. De Jong,et al.  Genetic algorithms as a tool for feature selection in machine learning , 1992, Proceedings Fourth International Conference on Tools with Artificial Intelligence TAI '92.

[13]  Eibe Frank,et al.  Large-scale attribute selection using wrappers , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[14]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[15]  Thomas Marill,et al.  On the effectiveness of receptors in recognition systems , 1963, IEEE Trans. Inf. Theory.

[16]  Michel Gendreau,et al.  Metaheuristics in Combinatorial Optimization , 2022 .

[17]  Byung Ro Moon,et al.  Hybrid Genetic Algorithms for Feature Selection , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Graham Kendall,et al.  Automatic heuristic generation with genetic programming: evolving a jack-of-all-trades or a master of one , 2007, GECCO '07.

[19]  Thomas G. Dietterich,et al.  Learning with Many Irrelevant Features , 1991, AAAI.

[20]  Rich Caruana,et al.  Greedy Attribute Selection , 1994, ICML.

[21]  Huan Liu,et al.  Efficient Feature Selection via Analysis of Relevance and Redundancy , 2004, J. Mach. Learn. Res..

[22]  Kourosh Neshatian,et al.  Feature Manipulation with Genetic Programming , 2010 .

[23]  Graham Kendall,et al.  Exploring Hyper-heuristic Methodologies with Genetic Programming , 2009 .

[24]  Nasser R. Sabar,et al.  A constructive hyper-heuristics for rough set attribute reduction , 2010, 2010 10th International Conference on Intelligent Systems Design and Applications.

[25]  Graham Kendall,et al.  Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques , 2013 .

[26]  Mark Johnston,et al.  A genetic programming based hyper-heuristic approach for combinatorial optimisation , 2011, GECCO '11.