Multi Objective Genetic Programming for Feature Construction in Classification Problems

This work introduces a new technique for features construction in classification problems by means of multi objective genetic programming (MOGP). The final goal is to improve the generalization ability of the final classifier. MOGP can help in finding solutions with a better generalization ability with respect to standard genetic programming as stated in [1]. The main issue is the choice of the criteria that must be optimized by MOGP. In this work the construction of new features is guided by two criteria: the first one is the entropy of the target classes as in [7] while the second is inspired by the concept of margin used in support vector machines.

[1]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[2]  Krzysztof Krawiec,et al.  Genetic Programming-based Construction of Features for Machine Learning and Knowledge Discovery Tasks , 2002, Genetic Programming and Evolvable Machines.

[3]  Yang Zhang,et al.  Domain-independent feature extraction for multi-classification using multi-objective genetic programming , 2010, Pattern Analysis and Applications.

[4]  Gary Geunbae Lee,et al.  Information gain and divergence-based feature selection for machine learning-based text categorization , 2006, Inf. Process. Manag..

[5]  Charles F. Hockett,et al.  A mathematical theory of communication , 1948, MOCO.

[6]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[7]  Mark Johnston,et al.  Feature Construction and Dimension Reduction Using Genetic Programming , 2007, Australian Conference on Artificial Intelligence.

[8]  Larry Bull,et al.  Genetic Programming with a Genetic Algorithm for Feature Construction and Selection , 2005, Genetic Programming and Evolvable Machines.

[9]  Mehmet A. Orgun,et al.  AI 2007: Advances in Artificial Intelligence, 20th Australian Joint Conference on Artificial Intelligence, Gold Coast, Australia, December 2-6, 2007, Proceedings , 2007, Australian Conference on Artificial Intelligence.

[10]  Riccardo Poli,et al.  A Field Guide to Genetic Programming , 2008 .

[11]  Alan G. Hawkes,et al.  A handbook of numerical and statistical techniques , 1977 .

[12]  Mengjie Zhang,et al.  Genetic Programming and Class-Wise Orthogonal Transformation for Dimension Reduction in Classification Problems , 2008, EuroGP.

[13]  Leonardo Vanneschi,et al.  A comparison of the generalization ability of different genetic programming frameworks , 2010, IEEE Congress on Evolutionary Computation.