A Survey of Optimization by Building and Using Probabilistic Models

This paper summarizes the research on population-based probabilistic search algorithms based on modeling promising solutions by estimating their probability distribution and using the constructed model to guide the exploration of the search space. It settles the algorithms in the field of genetic and evolutionary computation where they have been originated, and classifies them into a few classes according to the complexity of models they use. Algorithms within each class are briefly described and their strengths and weaknesses are discussed.

[1]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[2]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[3]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[4]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[5]  R. B. Ewen.  Nonparametric and Distribution-Free Methods , 1976 .

[6]  L. A. Marascuilo,et al.  Nonparametric and Distribution-Free Methods for the Social Sciences , 1977 .

[7]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[8]  Peter W. Sanders,et al.  Communication network optimization , 1988, Comput. Commun..

[9]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[10]  J. D. Schaffer,et al.  Real-Coded Genetic Algorithms and Interval-Schemata , 1992, FOGA.

[11]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[12]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[13]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[14]  Simon Handley,et al.  On the use of a directed acyclic graph to represent a population of computer programs , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[15]  Martin Pelikan,et al.  Hill Climbing with Learning (An Abstraction of Genetic Algorithm) , 1995 .

[16]  Heinz Mühlenbein,et al.  Fuzzy Recombination for the Breeder Genetic Algorithm , 1995, ICGA.

[17]  Kalyanmoy Deb,et al.  Simulated Binary Crossover for Continuous Search Space , 1995, Complex Syst..

[18]  Hans-Paul Schwefel,et al.  Parallel Problem Solving from Nature — PPSN IV , 1996, Lecture Notes in Computer Science.

[19]  versionStephan,et al.  Stochastic Hill Climbing with Learning by Vectors of NormalDistributionsSecond corrected and enhanced , 1996 .

[20]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[21]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[22]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[23]  Corso Elvezia.  Probabilistic Incremental Program Evolution , 1997 .

[24]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[25]  Louise Travé-Massuyès,et al.  Telephone Network Traffic Overloading Diagnosis and Evolutionary Computation Techniques , 1997, Artificial Evolution.

[26]  S. Baluja,et al.  Using Optimal Dependency-Trees for Combinatorial Optimization: Learning the Structure of the Search Space , 1997 .

[27]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[28]  Isao Ono,et al.  A Real Coded Genetic Algorithm for Function Optimization Using Unimodal Normal Distributed Crossover , 1997, ICGA.

[29]  Jürgen Schmidhuber,et al.  Probabilistic Incremental Program Evolution: Stochastic Search Through Program Space , 1997, ECML.

[30]  Shumeet Baluja,et al.  Using Optimal Dependency-Trees for Combinational Optimization , 1997, ICML.

[31]  Michèle Sebag,et al.  Extending Population-Based Incremental Learning to Continuous Search Spaces , 1998, PPSN.

[32]  Rajkumar Roy,et al.  Advances in Soft Computing: Engineering Design and Manufacturing , 1998 .

[33]  Heinz Mühlenbein,et al.  FDA -A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions , 1999, Evolutionary Computation.

[34]  Dirk Thierens,et al.  Linkage Information Processing In Distribution Estimation Algorithms , 1999, GECCO.

[35]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[36]  David E. Goldberg,et al.  The Gambler's Ruin Problem, Genetic Algorithms, and the Sizing of Populations , 1999, Evolutionary Computation.

[37]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[38]  Marcus Gallagher,et al.  Real-valued Evolutionary Optimization using a Flexible Probability Density Estimator , 1999, GECCO.

[39]  M. Pelikán,et al.  The Bivariate Marginal Distribution Algorithm , 1999 .

[40]  Heinz Mühlenbein,et al.  Schemata, Distributions and Graphical Models in Evolutionary Optimization , 1999, J. Heuristics.

[41]  G. Harik.  Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[42]  Franz Rothlauf,et al.  Bad Codings and the Utility of Well-Designed Genetic Algorithms , 2000, GECCO.

[43]  P. Bosman,et al.  Continuous iterated density estimation evolutionary algorithms within the IDEA framework , 2000 .

[44]  David E. Goldberg,et al.  Bayesian Optimization Algorithm, Population Sizing, and Time to Convergence , 2000, GECCO.

[45]  David E. Goldberg,et al.  Linkage Problem, Distribution Estimation, and Bayesian Networks , 2000, Evolutionary Computation.

[46]  David E. Goldberg,et al.  OMEGA - Ordering Messy GA: Solving Permutation Problems with the Fast Genetic Algorithm and Random Keys , 2000, GECCO.

[47]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.