Grammar-Based Genetic Programming with Bayesian network

Grammar-Based Genetic Programming (GBGP) improves the search performance of Genetic Programming (GP) by formalizing constraints and domain specific knowledge in grammar. The building blocks (i.e. the functions and the terminals) in a program can be dependent. Random crossover and mutation destroy the dependence with a high probability, hence breeding a poor program from good programs. Understanding on the syntactic and semantic in the grammar plays an important role to boost the efficiency of GP by reducing the number of poor breeding. Therefore, approaches have been proposed by introducing context sensitive ingredients encoded in probabilistic models. In this paper, we propose Grammar-Based Genetic Programming with Bayesian Network (BGBGP) which learns the dependence by attaching a Bayesian network to each derivation rule and demonstrates its effectiveness in two benchmark problems.

[1]  Hussein A. Abbass,et al.  AntTAG: a new method to compose computer programs using colonies of ants , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[2]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[3]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[4]  David E. Goldberg,et al.  Bayesian Optimization Algorithm: From Single Level to Hierarchy , 2002 .

[5]  Heinz Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions II. Continuous Parameters , 1996, PPSN.

[6]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[7]  Heinz Mühlenbein,et al.  FDA -A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions , 1999, Evolutionary Computation.

[8]  Kwong-Sak Leung,et al.  Combining genetic programming and inductive logic programming using logic grammars , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[9]  Peter A. Whigham,et al.  Grammatically-based Genetic Programming , 1995 .

[10]  Riccardo Poli,et al.  A Linear Estimation-of-Distribution GP System , 2008, EuroGP.

[11]  Aurora Trinidad Ramirez Pozo,et al.  Bayesian Automatic Programming , 2005, EuroGP.

[12]  Ray J. Solomonoff,et al.  Complexity-based induction systems: Comparisons and convergence theorems , 1978, IEEE Trans. Inf. Theory.

[13]  Martin Pelikan,et al.  Bayesian Optimization Algorithm , 2005 .

[14]  Hussein A. Abbass,et al.  Program evolution with explicit learning , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[15]  Yoshihiko Hasegawa.  Programming with Annotated Grammar Estimation , 2012 .

[16]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[17]  Hitoshi Iba,et al.  Binary encoding for prototype tree of probabilistic model building GP , 2009, GECCO.

[18]  M. Pelikán,et al.  The Bivariate Marginal Distribution Algorithm , 1999 .

[19]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1998, Learning in Graphical Models.

[20]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[21]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[22]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[23]  H. Iba,et al.  Estimation of distribution programming based on Bayesian network , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[24]  William F. Punch.  HOW EFFECTIVE ARE MULTIPLE POPULATIONS IN GENETIC PROGRAMMING , 1998 .

[25]  D. Goldberg,et al.  Probabilistic Model Building and Competent Genetic Programming , 2003 .

[26]  Peter A. Whigham,et al.  Grammar-based Genetic Programming: a survey , 2010, Genetic Programming and Evolvable Machines.

[27]  Rafal Salustowicz,et al.  Probabilistic Incremental Program Evolution , 1997, Evolutionary Computation.

[28]  Hussein A. Abbass,et al.  Grammar model-based program evolution , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[29]  Hitoshi Iba,et al.  Estimation of Bayesian Network for Program Generation , 2006 .

[30]  Peter A. N. Bosman,et al.  Grammar Transformations in an EDA for Genetic Programming , 2004 .

[31]  John R. Koza,et al.  Genetic programming as a means for programming computers by natural selection , 1994 .

[32]  Walter Böhm,et al.  Exact Uniform Initialization For Genetic Programming , 1996, FOGA.

[33]  Nguyen Xuan Hoai,et al.  Probabilistic model building in genetic programming: a critical review , 2013, Genetic Programming and Evolvable Machines.

[34]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[35]  S. Baluja,et al.  Combining Multiple Optimization Runs with Optimal Dependency Trees , 1997 .

[36]  Hagai Attias,et al.  Inferring Parameters and Structure of Latent Variable Models by Variational Bayes , 1999, UAI.

[37]  Taylor L. Booth,et al.  Applying Probability Measures to Abstract Languages , 1973, IEEE Transactions on Computers.

[38]  Hitoshi Iba,et al.  Latent Variable Model for Estimation of Distribution Algorithm Based on a Probabilistic Context-Free Grammar , 2009, IEEE Transactions on Evolutionary Computation.

[39]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[40]  Hitoshi Iba,et al.  A Bayesian Network Approach to Program Generation , 2008, IEEE Transactions on Evolutionary Computation.

[41]  Ben Goertzel,et al.  Learning computer programs with the bayesian optimization algorithm , 2005, GECCO '05.

[42]  Michèle Sebag,et al.  Avoiding the Bloat with Stochastic Grammar-Based Genetic Programming , 2001, Artificial Evolution.

[43]  Danushka Bollegala,et al.  Probabilistic model building GP with Belief propagation , 2012, 2012 IEEE Congress on Evolutionary Computation.

[44]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[45]  Ivan Tanev,et al.  Incorporating Learning Probabilistic Context-Sensitive Grammar in Genetic Programming for Efficient Evolution and Adaptation of Snakebot , 2005, EuroGP.