Reducing Bloat in GP with Multiple Objectives

This chapter investigates the use of multiobjective techniques in genetic programming (GP) in order to evolve compact programs and to reduce the effects caused by bloating. The underlying approach considers the program size as a second, independent objective besides program functionality, and several studies have found this concept to be successful in reducing bloat. Based on one specific algorithm, we demonstrate the principle of multiobjective GP and show how to apply Pareto-based strategies to GP. This approach outperforms four classical strategies to reduce bloat with regard to both convergence speed and size of the produced programs on an even-parity problem. Additionally, we investigate the question of why the Pareto-based strategies can be more effective in reducing bloat than alternative strategies on several test problems. The analysis falsifies the hypothesis that the small but less functional individuals that are kept in the population act as building blocks building blocks for larger correct solutions. This leads to the conclusion that the advantages are probably due to the increased diversity in the population.

[1]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[2]  Anikó Ekárt,et al.  Selection Based on the Pareto Nondomination Criterion for Controlling Code Growth in Genetic Programming , 2001, Genetic Programming and Evolvable Machines.

[3]  Richard A. Watson,et al.  Reducing Local Optima in Single-Objective Problems by Multi-objectivization , 2001, EMO.

[4]  Matthew J. Streeter,et al.  Automated Discovery of Numerical Approximation Formulae via Genetic Programming , 2001, Genetic Programming and Evolvable Machines.

[5]  Tatiana Kalganova,et al.  Evolving more efficient digital circuits by allowing circuit layout evolution and multi-objective fitness , 1999, Proceedings of the First NASA/DoD Workshop on Evolvable Hardware.

[6]  William B. Langdon,et al.  Quadratic Bloat in Genetic Programming , 2000, GECCO.

[7]  Hussein A. Abbass,et al.  Searching under Multi-evolutionary Pressures , 2003, EMO.

[8]  Mark Kotanchek,et al.  Pareto-Front Exploitation in Symbolic Regression , 2005 .

[9]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[10]  Riccardo Poli,et al.  Fitness Causes Bloat , 1998 .

[11]  Lothar Thiele,et al.  Multiobjective genetic programming: reducing bloat using SPEA2 , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[12]  Xiaodong Li,et al.  Multiobjective parsimony enforcement for superior generalisation performance , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[13]  Arthur K. Kordon,et al.  Biomass Inferential Sensor Based on Ensemble of Models Generated by Genetic Programming , 2004, GECCO.

[14]  David E. Goldberg,et al.  Accuracy, Parsimony, and Generality in Evolutionary Learning Systems via Multiobjective Selection , 2002, IWLCS.

[15]  John R. Koza,et al.  Genetic programming 2 - automatic discovery of reusable programs , 1994, Complex Adaptive Systems.

[16]  Eckart Zitzler,et al.  Evolutionary algorithms for multiobjective optimization: methods and applications , 1999 .

[17]  Sean Luke,et al.  Alternative Bloat Control Methods , 2004, GECCO.

[18]  Marco Laumanns,et al.  SPEA2: Improving the Strength Pareto Evolutionary Algorithm For Multiobjective Optimization , 2002 .

[19]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[20]  Mark Kotanchek,et al.  Pursuing the Pareto Paradigm: Tournaments, Algorithm Variations and Ordinal Optimization , 2007 .

[21]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[22]  Xiaodong Li,et al.  Multi-objective techniques in genetic programming for evolving classifiers , 2005, 2005 IEEE Congress on Evolutionary Computation.

[23]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[24]  Edwin D. de Jong,et al.  Multi-Objective Methods for Tree Size Control , 2003, Genetic Programming and Evolvable Machines.

[25]  Terence Soule,et al.  Effects of Code Growth and Parsimony Pressure on Populations in Genetic Programming , 1998, Evolutionary Computation.

[26]  Tobias Blickle,et al.  Evolving Compact Solutions in Genetic Programming: A Case Study , 1996, PPSN.

[27]  Wolfgang Banzhaf,et al.  Genetic Programming: An Introduction , 1997 .

[28]  Byoung-Tak Zhang,et al.  Balancing Accuracy and Parsimony in Genetic Programming , 1995, Evolutionary Computation.

[29]  Andrew Hunter,et al.  Expression Inference - Genetic Symbolic Classification Integrated with Non-linear Coefficient Optimisation , 2002, AISC.

[30]  Yang Zhang,et al.  Evolving optimal feature extraction using multi-objective genetic programming: a methodology and preliminary study on edge detection , 2005, GECCO '05.

[31]  Edwin D. de Jong,et al.  Reducing bloat and promoting diversity using multi-objective methods , 2001 .

[32]  Terence Soule,et al.  Removal bias: a new cause of code growth in tree based evolutionary programming , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).