Explicitly Simplifying Evolved Genetic Programs during Evolution

The genetic programming (GP) evolutionary process typically introduces a large amount of redundancy and unnecessary complexity into evolved programs. Quick growth of redundant and functionally useless sections of programs can quickly overcome a GP system, exhausting system resources and causing premature termination of the system before an acceptable solution can be found. Rather than implicitly controlling the redundancy and code growth/bloat as in most of the existing approaches, this paper investigates an algebraic simplification algorithm for explicitly removing the redundancy from the genetic programs and simplifying these programs online during the evolutionary process. The new GP system with the simplification is examined and compared with a standard GP system on two regression and three classification problems of varying difficulties. The results show that the GP system employing a simplification component can achieve superior efficiency with comparable or slightly superior effectiveness to the standard GP system on these problems. The programs evolved by the new GP approach with the explicit simplification contain ``hidden patterns'' for a particular problem and are relatively simple and easy to interpret.

[1]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Wolfgang Banzhaf,et al.  Automatic Generation of Control Programs for Walking Robots Using Genetic Programming , 2002, EuroGP.

[3]  Stefano Cagnoni,et al.  Introduction to the special issue on evolutionary computer vision and image understanding , 2006, Pattern Recognit. Lett..

[4]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[5]  Riccardo Poli,et al.  Fitness Causes Bloat , 1998 .

[6]  Graham Kendall,et al.  Problem Difficulty and Code Growth in Genetic Programming , 2004, Genetic Programming and Evolvable Machines.

[7]  Peter Nordin,et al.  Complexity Compression and Evolution , 1995, ICGA.

[8]  Anikó Ekárt,et al.  Shorter Fitness Preserving Genetic Programs , 1999, Artificial Evolution.

[9]  Mengjie Zhang,et al.  Multiclass Object Classification Using Genetic Programming , 2004, EvoWorkshops.

[10]  Martin Anthony,et al.  PAC learning and neural networks , 1998 .

[11]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[12]  James F. Smith,et al.  Evolving fuzzy decision tree structure that adapts in real-time , 2005, GECCO '05.

[13]  R. Solomonoff.  A SYSTEM FOR INCREMENTAL LEARNING BASED ON ALGORITHMIC PROBABILITY , 1989 .

[14]  William B. Langdon,et al.  Quadratic Bloat in Genetic Programming , 2000, GECCO.

[15]  Lothar Thiele,et al.  Genetic Programming and Redundancy , 1994 .

[16]  Sean Luke,et al.  Code Growth Is Not Caused by Introns , 2000 .

[17]  Terence Soule,et al.  Growth of self-canceling code in evolutionary systems , 2006, GECCO.

[18]  Sean Luke,et al.  Lexicographic Parsimony Pressure , 2002, GECCO.

[19]  Vic Ciesielski,et al.  Representing classification problems in genetic programming , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[20]  Riccardo Poli,et al.  Fitness Causes Bloat: Mutation , 1997, EuroGP.

[21]  Nicholas S. Flann,et al.  Improving the accuracy and robustness of genetic programming through expression simplification , 1996 .

[22]  Matthew J. Streeter,et al.  The Root Causes of Code Growth in Genetic Programming , 2003, EuroGP.

[23]  Victor Ciesielski,et al.  Genetic Programming for Multiple Class Object Detection , 1999, Australian Joint Conference on Artificial Intelligence.

[24]  Mengjie Zhang,et al.  Program Size and Pixel Statistics in Genetic Programming for Object Detection , 2004, EvoWorkshops.

[25]  M. L. Berenson,et al.  Applied Statistics: A First Course. , 1989 .

[26]  Terence Soule,et al.  Code growth in genetic programming , 1996 .

[27]  Xiaodong Li,et al.  Multi-objective techniques in genetic programming for evolving classifiers , 2005, 2005 IEEE Congress on Evolutionary Computation.

[28]  Ernesto Costa,et al.  Comparing tree depth limits and resource-limited GP , 2005, 2005 IEEE Congress on Evolutionary Computation.

[29]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[30]  Annie S. Wu,et al.  Winnowing Wheat from Chaff: The Chunking GA , 2004, GECCO.

[31]  Nicholas Freitag McPhee,et al.  Accurate Replication in Genetic Programming , 1995, ICGA.

[32]  Terence Soule,et al.  Effects of Code Growth and Parsimony Pressure on Populations in Genetic Programming , 1998, Evolutionary Computation.

[33]  James F. Smith.  Genetic Program Based Data Mining for Fuzzy Decision Trees , 2004, IDEAL.

[34]  William B. Langdon,et al.  Fitness Causes Bloat: Simulated Annealing, Hill Climbing and Populations , 1997 .

[35]  Byoung-Tak Zhang,et al.  Balancing Accuracy and Parsimony in Genetic Programming , 1995, Evolutionary Computation.

[36]  Gaston H. Gonnet,et al.  Determining equivalence of expressions in random polynomial time , 1984, STOC '84.

[37]  Harald Niederreiter,et al.  Introduction to finite fields and their applications: Preface , 1994 .

[38]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[39]  Ray J. Solomonoff,et al.  The Application of Algorithmic Probability to Problems in Artificial Intelligence , 1985, UAI.

[40]  David Jackson.  Fitness evaluation avoidance in Boolean GP problems , 2005, 2005 IEEE Congress on Evolutionary Computation.

[41]  William D. Smart,et al.  Program Simplification in Genetic Programming for Object Classification , 2005, KES.

[42]  Peter J. Angeline,et al.  Explicitly Defined Introns and Destructive Crossover in Genetic Programming , 1996 .