Representation and structural difficulty in genetic programming

Standard tree-based genetic programming suffers from a structural difficulty problem in that it is unable to search effectively for solutions requiring very full or very narrow trees. This deficiency has been variously explained as a consequence of restrictions imposed by the tree structure or as a result of the numerical distribution of tree shapes. We show that by using a different tree-based representation and local (insertion and deletion) structural modification operators, that this problem can be almost eliminated even with trivial (stochastic hill-climbing) search methods, thus eliminating the above explanations. We argue, instead, that structural difficulty is a consequence of the large step size of the operators in standard genetic programming, which is itself a consequence of the fixed-arity property embodied in its representation.

[1]  W. Rudin.  Principles of mathematical analysis , 1964 .

[2]  Jason M. Daida,et al.  What Makes a Problem GP-Hard? Analysis of a Tunably Difficult Problem in Genetic Programming , 1999, Genetic Programming and Evolvable Machines.

[3]  Aravind K. Joshi,et al.  Tree-Adjoining Grammars , 1997, Handbook of Formal Languages.

[4]  Inman Harvey,et al.  The Use of Neutral Genotype-Phenotype Mappings for Improved Evolutionary Search , 2000 .

[5]  Jason M. Daida,et al.  Challenges with Verification, Repeatability, and Meaningful Comparisons in Genetic Programming , 1997 .

[6]  Wolfgang Banzhaf,et al.  Genetic Programming: An Introduction , 1997 .

[7]  Per Kristian Lehre,et al.  Developmental mappings and phenotypic complexity , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[8]  Franz Rothlauf,et al.  Representations for genetic and evolutionary algorithms , 2002, Studies in Fuzziness and Soft Computing.

[9]  Riccardo Poli,et al.  Foundations of Genetic Programming , 1999, Springer Berlin Heidelberg.

[10]  Jason M. Daida,et al.  Analysis of single-node (building) blocks in genetic programming , 1999 .

[11]  Richard C. Waters,et al.  Tree Insertion Grammar: A Cubic-Time, Parsable Formalism that Lexicalizes Context-Free Grammar without Changing the Trees Produced , 1995, CL.

[12]  Günther R. Raidl,et al.  Characterizing Locality in Decoder-Based EAs for the Multidimensional Knapsack Problem , 1999, Artificial Evolution.

[13]  Terence Soule,et al.  Effects of Code Growth and Parsimony Pressure on Populations in Genetic Programming , 1998, Evolutionary Computation.

[14]  C. Igel,et al.  Structure optimization and isomorphisms , 2001 .

[15]  Nguyen Xuan Hoai,et al.  Genetic Transposition in Tree-Adjoining Grammar Guided Genetic Programming: The Duplication Operator , 2005, EuroGP.

[16]  Dirk Wiesmann,et al.  On Representation and Genetic Operators in Evolutionary Algorithms , 1998 .

[17]  Shin-Yee Lu.  A Tree-to-Tree Distance and Its Application to Cluster Analysis , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  C. Igel.  Causality of hierarchical variable length representations , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[19]  Una-May O’Reilly.  Using a distance metric on genetic programs to understand genetic operators , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[20]  Nguyen Xuan Hoai,et al.  Some Experimental Results with Tree Adjunct Grammar Guided Genetic Programming , 2002, EuroGP.

[21]  Charles C. Palmer,et al.  Representing trees in genetic algorithms , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[22]  Anthony S. Kroch,et al.  Unbounded Dependencies and Subjacency in a Tree Adjoining Grammar , 1987 .

[23]  Jason M. Daida,et al.  What Makes a Problem GP-Hard? Validating a Hypothesis of Structural Causes , 2003, GECCO.

[24]  Justinian P. Rosca,et al.  Causality in Genetic Programming , 1995, International Conference on Genetic Algorithms.

[25]  Lothar Thiele,et al.  Genetic Programming and Redundancy , 1994 .

[26]  Philippe Flajolet,et al.  An introduction to the analysis of algorithms , 1995 .

[27]  Aravind K. Joshi,et al.  Mathematical and computational aspects of lexicalized grammars , 1990 .

[28]  M. Shackleton,et al.  An investigation of redundant genotype-phenotype mappings and their role in evolutionary search , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[29]  Jason M. Daida,et al.  Limits to expression in genetic programming: lattice-aggregate modeling , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[30]  Stuart M. Shieber,et al.  An Alternative Conception of Tree-Adjoining Derivation , 1992, ACL.

[31]  Aravind K. Joshi,et al.  An Introduction to Tree Adjoining Grammar , 1987 .

[32]  Nguyen Xuan Hoai,et al.  GENETIC TRANPOSITION IN TREE-ADJOINING GRAMMAR GUIDED GENTIC PROGRAMMING : THE RELOCATION OPERATOR , 2007 .

[33]  Richard C. Waters,et al.  A Cubic-Time Parsable , Lexicalized Normal Form For Context-Free Grammar That Preserves Tree Structure , 1993 .

[34]  Aravind K. Joshi,et al.  Constraints on Structural Descriptions: Local Transformations , 1977, SIAM J. Comput..

[35]  Aravind K. Joshi,et al.  A study of tree adjoining grammars , 1987 .

[36]  Aravind K. Joshi,et al.  Tree Adjunct Grammars , 1975, J. Comput. Syst. Sci..

[37]  David J. Weir,et al.  Characterizing mildly context-sensitive grammar formalisms , 1988 .

[38]  R. Shipman,et al.  Genetic Redundancy: Desirable or Problematic for Evolutionary Adaptation? , 1999, ICANNGA.

[39]  Bernhard Sendhoff,et al.  A Condition for the Genotype-Phenotype Mapping: Causality , 1997, ICGA.

[40]  Nichael Lynn Cramer,et al.  A Representation for the Adaptive Generation of Simple Sequential Programs , 1985, ICGA.

[41]  David J. Weir,et al.  The convergence of mildly context-sensitive grammar formalisms , 1990 .

[42]  Anthony S. Kroch,et al.  The Linguistic Relevance of Tree Adjoining Grammar , 1985 .

[43]  Richard C. Waters,et al.  Lexicalized Context-Free Grammars , 1993, ACL.

[44]  Rachel Harrison,et al.  Characterizing a Tunably Difficult Problem in Genetic Programming , 2000, GECCO.

[45]  Peter A. Whigham,et al.  Grammatical bias for evolutionary learning , 1996 .