Grammar model-based program evolution

In evolutionary computation, genetic operators, such as mutation and crossover, are employed to perturb individuals to generate the next population. However these fixed, problem independent genetic operators may destroy the sub-solution, usually called building blocks, instead of discovering and preserving them. One way to overcome this problem is to build a model based on the good individuals, and sample this model to obtain the next population. There is a wide range of such work in genetic algorithms; but because of the complexity of the genetic programming (GP) tree representation, little work of this kind has been done in GP. In this paper, we propose a new method, grammar model-based program evolution (GMPE) to evolved GP program. We replace common GP genetic operators with a probabilistic context-free grammar (SCFG). In each generation, an SCFG is learnt, and a new population is generated by sampling this SCFG model. On two benchmark problems we have studied, GMPE significantly outperforms conventional GP, learning faster and more reliably.

[1]  Andreas Stolcke,et al.  Bayesian learning of probabilistic language models , 1994 .

[2]  Wray L. Buntine,et al.  A theory of learning classification rules , 1990 .

[3]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[4]  Man Leung Wong,et al.  Evolutionary Program Induction Directed by Logic Grammars , 1997, Evolutionary Computation.

[5]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[6]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[7]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[8]  Jorma Rissanen,et al.  Stochastic Complexity in Statistical Inquiry , 1989, World Scientific Series in Computer Science.

[9]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[10]  Hussein A. Abbass,et al.  AntTAG: a new method to compose computer programs using colonies of ants , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[11]  C. S. Wallace,et al.  An Information Measure for Classification , 1968, Comput. J..

[12]  TALFOURD ELY,et al.  University College , 1878, Nature.

[13]  Hussein A. Abbass,et al.  Program Evolution with Explicit Learning: a New Framework for Program Automatic Synthesis , 2003 .

[14]  H. Muhlenbein,et al.  The Factorized Distribution Algorithm for additively decomposed functions , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[15]  David E. Goldberg,et al.  Bayesian Optimization Algorithm: From Single Level to Hierarchy , 2002 .

[16]  Erik D. Goodman,et al.  The royal tree problem, a benchmark for single and multiple population genetic programming , 1996 .

[17]  H. Iba,et al.  Estimation of distribution programming based on Bayesian network , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[18]  D. Goldberg,et al.  Probabilistic Model Building and Competent Genetic Programming , 2003 .

[19]  Fernando G. Lobo,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[20]  W. Langdon.  An Analysis of the MAX Problem in Genetic Programming , 1997 .

[21]  Michael O'Neill,et al.  Grammatical Evolution: Evolving Programs for an Arbitrary Language , 1998, EuroGP.

[22]  Stanley F. Chen,et al.  Bayesian Grammar Induction for Language Modeling , 1995, ACL.

[23]  C. S. Wallace,et al.  Estimation and Inference by Compact Coding , 1987 .

[24]  S. Baluja,et al.  Using Optimal Dependency-Trees for Combinatorial Optimization: Learning the Structure of the Search Space , 1997 .

[25]  Ryszard S. Michalski,et al.  LEARNABLE EVOLUTION MODEL: Evolutionary Processes Guided by Machine Learning , 2004, Machine Learning.

[26]  David L. Dowe,et al.  Minimum Message Length and Kolmogorov Complexity , 1999, Comput. J..

[27]  Peter A. Whigham,et al.  Grammatically-based Genetic Programming , 1995 .

[28]  M. Pelikán,et al.  The Bivariate Marginal Distribution Algorithm , 1999 .

[29]  Peter A. Whigham.  Inductive bias and genetic programming , 1995 .

[30]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[31]  Hussein A. Abbass,et al.  Program evolution with explicit learning , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[32]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[33]  G. Harik.  Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[34]  Peter J. Angeline,et al.  On Using Syntactic Constraints with Genetic Programming , 1996 .

[35]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[36]  Rafal Salustowicz,et al.  Probabilistic Incremental Program Evolution , 1997, Evolutionary Computation.