Estimation of distribution algorithm based on probabilistic grammar with latent annotations

Genetic Programming (GP) which mimics the natural evolution to optimize functions and programs, has been applied to many problems. In recent years, evolutionary algorithms are seen from the viewpoint of the estimation of distribution. Many algorithms called EDAs (Estimation of Distribution Algorithms) based on probabilistic techniques have been proposed. Although probabilistic context free grammar (PCFG) is often used for the function and program evolution, it assumes the independence among the production rules. With this simple PCFG, it is not able to induce the building-blocks from promising solutions. We have proposed a new function evolution algorithm based on PCFG using latent annotations which weaken the independence assumption. Computational experiments on two subjects (the royal tree problem and the DMAX problem) demonstrate that our new approach is highly effective compared to prior approaches.

[1]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[2]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[3]  Hitoshi Iba,et al.  Optimizing Programs with Estimation of Bayesian Network , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[4]  Melanie Mitchell,et al.  The royal road for genetic algorithms: Fitness landscapes and GA performance , 1991 .

[5]  Ben Goertzel,et al.  Learning computer programs with the bayesian optimization algorithm , 2005, GECCO '05.

[6]  P. Ross,et al.  An adverse interaction between crossover and restricted tree depth in genetic programming , 1996 .

[7]  Hussein A. Abbass,et al.  Program Evolution with Explicit Learning: a New Framework for Program Automatic Synthesis , 2003 .

[8]  Peter A. N. Bosman,et al.  Grammar Transformations in an EDA for Genetic Programming , 2004 .

[9]  Corso Elvezia.  Probabilistic Incremental Program Evolution , 1997 .

[10]  Una-May O'Reilly,et al.  Genetic Programming II: Automatic Discovery of Reusable Programs. , 1994, Artificial Life.

[11]  Petr Posík.  Estimation of Distribution Algorithms , 2006 .

[12]  Peter A. Whigham,et al.  Grammatically-based Genetic Programming , 1995 .

[13]  Rafal Salustowicz,et al.  Probabilistic Incremental Program Evolution , 1997, Evolutionary Computation.

[14]  Hussein A. Abbass,et al.  Grammar model-based program evolution , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[15]  H. Muhlenbein,et al.  The Factorized Distribution Algorithm for additively decomposed functions , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[16]  Hitoshi Iba,et al.  Estimation of Bayesian Network for Program Generation , 2006 .

[17]  Hitoshi Iba,et al.  Probabilistic distribution models for EDA-based GP , 2005, GECCO '05.

[18]  W. Langdon.  An Analysis of the MAX Problem in Genetic Programming , 1997 .

[19]  Aurora Trinidad Ramirez Pozo,et al.  Bayesian Automatic Programming , 2005, EuroGP.

[20]  Jun'ichi Tsujii,et al.  Probabilistic CFG with Latent Annotations , 2005, ACL.

[21]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[22]  G. Harik.  Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[23]  H. Iba,et al.  Estimation of distribution programming based on Bayesian network , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[24]  William F. Punch.  HOW EFFECTIVE ARE MULTIPLE POPULATIONS IN GENETIC PROGRAMMING , 1998 .

[25]  D. Goldberg,et al.  Probabilistic Model Building and Competent Genetic Programming , 2003 .