A unified Bayesian framework for evolutionary learning and optimization

A probabilistic evolutionary framework is presented and shown to be applicable to both learning and optimization problems. In this framework, evolutionary computation is viewed as Bayesian inference that iteratively updates the posterior distribution of a population from the prior knowledge and observation of new individuals to find an individual with the maximum posterior probability. Theoretical foundations of Bayesian evolutionary computation are given and its generality is demonstrated by showing specific Bayesian evolutionary algorithms for learning and optimization. We also discuss how the probabilistic framework can be used to develop novel evolutionary algorithms that embed evolutionary learning for evolutionary optimization and vice versa.

[1]  Byoung-Tak Zhang,et al.  Convergence properties of incremental Bayesian evolutionary algorithms with single Markov chains , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[2]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[3]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[4]  Heinz Mühlenbein,et al.  Schemata, Distributions and Graphical Models in Evolutionary Optimization , 1999, J. Heuristics.

[5]  D. Fogel.  Evolutionary algorithms in theory and practice , 1997, Complex..

[6]  Pat Langley,et al.  Editorial: On Machine Learning , 1986, Machine Learning.

[7]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[8]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[9]  Geoffrey E. Hinton,et al.  The "wake-sleep" algorithm for unsupervised neural networks. , 1995, Science.

[10]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[11]  Byoung-Tak Zhang.  A Bayesian framework for evolutionary computation , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[12]  Hans-Paul Schwefel,et al.  Evolution and Optimum Seeking: The Sixth Generation , 1993 .

[13]  J. Rissanen.  Stochastic Complexity and Modeling , 1986 .

[14]  David B. Fogel,et al.  Using fitness distributions to design more efficient evolutionary computations , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[15]  David B. Fogel,et al.  Evolutionary Computation: The Fossil Record , 1998 .

[16]  Richard M. Friedberg,et al.  A Learning Machine: Part I , 1958, IBM J. Res. Dev..

[17]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[18]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[19]  Byoung-Tak Zhang,et al.  Evolving neural trees for time series prediction using Bayesian evolutionary algorithms , 2000, 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks. Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (Cat. No.00.

[20]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[21]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[22]  Byoung-Tak Zhang,et al.  Bayesian Methods for Efficient Genetic Programming , 2000, Genetic Programming and Evolvable Machines.

[23]  Peter Nordin,et al.  Genetic programming - An Introduction: On the Automatic Evolution of Computer Programs and Its Applications , 1998 .

[24]  Shumeet Baluja,et al.  Using Optimal Dependency-Trees for Combinational Optimization , 1997, ICML.

[25]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[26]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[27]  M. Pelikán,et al.  The Bivariate Marginal Distribution Algorithm , 1999 .

[28]  Rajkumar Roy,et al.  Advances in Soft Computing: Engineering Design and Manufacturing , 1998 .

[29]  Byoung-Tak Zhang,et al.  Evolutionary Induction of Sparse Neural Trees , 1997, Evolutionary Computation.

[30]  S. Baluja,et al.  Using Optimal Dependency-Trees for Combinatorial Optimization: Learning the Structure of the Search Space , 1997 .

[31]  Robert A. Lordo,et al.  Learning from Data: Concepts, Theory, and Methods , 2001, Technometrics.

[32]  Heinz Mühlenbein,et al.  FDA -A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions , 1999, Evolutionary Computation.

[33]  Wolfgang Banzhaf,et al.  Genetic Programming: An Introduction , 1997 .

[34]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[35]  Rich Caruana,et al.  Removing the Genetics from the Standard Genetic Algorithm , 1995, ICML.

[36]  Byoung-Tak Zhang,et al.  Bayesian Evolutionary Optimization Using Helmholtz Machines , 2000, PPSN.