Estimation of Distribution Algorithms Applied To Combinatorial Optimization Problems

Estimation of Distribution Algorithms (EDAs) are a new tool for Evolutionary Computation. Based on Genetic Algorithms (GAs) this new class of algorithms generalizes GAs by replacing the crossover and mutation operators by learning and sampling the probability distribution of the best individuals of the population at each iteration. In this paper we present an introduction to EDAs in the field of combinatorial optimization. The algorithms are organised taking the complexity of the probabilistic model used into account. We also provide some points to the literature.

[1]  Thilo Mahnig,et al.  Mathematical Analysis of Optimization Methods Using Search Distributions , 2000 .

[2]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[3]  Rafal Salustowicz,et al.  Probabilistic Incremental Program Evolution , 1997, Evolutionary Computation.

[4]  E. Forgy.  Cluster analysis of multivariate data : efficiency versus interpretability of classifications , 1965 .

[5]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[6]  H. Muhlenbein,et al.  The Factorized Distribution Algorithm for additively decomposed functions , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[7]  David E. Goldberg,et al.  Linkage Problem, Distribution Estimation, and Bayesian Networks , 2000, Evolutionary Computation.

[8]  Martin Pelikan,et al.  Hill Climbing with Learning (An Abstraction of Genetic Algorithm) , 1995 .

[9]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[10]  Jiri Ocenasek,et al.  EXPERIMENTAL STUDY: HYPERGRAPH PARTITIONING BASED ON THE SIMPLE AND ADVANCED GENETIC ALGORITHM BMDA AND BOA , 2002 .

[11]  Pedro Larrañaga,et al.  Parallel Estimation of Distribution Algorithms , 2002, Estimation of Distribution Algorithms.

[12]  S. Baluja,et al.  Combining Multiple Optimization Runs with Optimal Dependency Trees , 1997 .

[13]  Heinz Mühlenbein,et al.  FDA -A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions , 1999, Evolutionary Computation.

[14]  Marcus Gallagher,et al.  Multi-layer Perceptron Error Surfaces: Visualization, Structure and Modelling , 2000 .

[15]  S. Baluja,et al.  Using Optimal Dependency-Trees for Combinatorial Optimization: Learning the Structure of the Search Space , 1997 .

[16]  Dirk Thierens,et al.  Linkage Information Processing In Distribution Estimation Algorithms , 1999, GECCO.

[17]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[18]  Ivo Everts,et al.  Extended Compact Genetic Algorithm , 2004 .

[19]  David E. Goldberg,et al.  SEARCH, Blackbox Optimization, And Sample Complexity , 1996, FOGA.

[20]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[21]  David E. Goldberg,et al.  Bayesian Optimization Algorithm, Population Sizing, and Time to Convergence , 2000, GECCO.

[22]  David E. Goldberg,et al.  Bayesian optimization algorithm, decision graphs, and Occam's razor , 2001 .

[23]  A. Juels,et al.  Topics in black-box combinatorial optimization , 1996 .

[24]  Luis M. de Campos,et al.  Partial Abductive Inference in Bayesian Networks: An Empirical Comparison Between GAs and EDAs , 2002, Estimation of Distribution Algorithms.

[25]  M. Schmidt,et al.  Adding genetics to the standard PBIL algorithm , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[26]  Sven Anderson,et al.  Training hidden Markov Models using population-based learning , 1999 .

[27]  Hillol Kargupta,et al.  The Gene Expression Messy Genetic Algorithm , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[28]  Shumeet Baluja,et al.  Genetic Algorithms and Explicit Search Statistics , 1996, NIPS.

[29]  Roberto Santana,et al.  The Factorized Distribution Algorithm and The Junction Tree: A Learning Perspective , 2005 .

[30]  A. A. Zhigli︠a︡vskiĭ,et al.  Theory of Global Random Search , 1991 .

[31]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[32]  Erick Cantú-Paz,et al.  Feature Subset Selection by Estimation of Distribution Algorithms , 2002, GECCO.

[33]  D. Goldberg,et al.  Research on the Bayesian Optimization Algorithm Research on the Bayesian Optimization Algorithm , 2022 .

[34]  G. Harik.  Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[35]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[36]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[37]  Pedro Larrañaga,et al.  Feature Subset Selection by Bayesian network-based optimization , 2000, Artif. Intell..

[38]  Larry J. Eshelman,et al.  Productive Recombination and Propagating and Preserving Schemata , 1994, FOGA.

[39]  Rahul Sukthankar,et al.  Evolving an intelligent vehicle for tactical reasoning in traffic , 1997, Proceedings of International Conference on Robotics and Automation.

[40]  Rafal Salustowicz,et al.  Learning to Predict Through Probabilistic Incremental Program Evolution and Automatic Task Decomposition , 1998 .

[41]  ROBERTO SANTANA,et al.  FINDING TYPICAL TESTORS BY USING AN EVOLUTIONARY STRATEGY EDUARDO ALBA-CABRERA , .

[42]  Pedro Larrañaga,et al.  The Convergence Behavior of the PBIL Algorithm: A Preliminary Approach , 2001 .

[43]  Heinz Mühlenbein,et al.  Schemata, Distributions and Graphical Models in Evolutionary Optimization , 1999, J. Heuristics.

[44]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[45]  M. Servais,et al.  Function Optimisation Using Multiple-base Population Based Incremental Learning , 1997 .

[46]  Shumeet Baluja,et al.  Fast Probabilistic Modeling for Combinatorial Optimization , 1998, AAAI/IAAI.

[47]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[48]  Byoung-Tak Zhang,et al.  Evolving neural trees for time series prediction using Bayesian evolutionary algorithms , 2000, 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks. Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (Cat. No.00.

[49]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[50]  Markus Höhfeld,et al.  Improving the Generalization Performance of Multi-Layer-Perceptrons with Population-Based Incremental Learning , 1996, PPSN.

[51]  Byoung-Tak Zhang.  A Bayesian framework for evolutionary computation , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[52]  David E. Goldberg,et al.  Genetic Algorithms, Clustering, and the Breaking of Symmetry , 2000, PPSN.

[53]  T. Mahnig,et al.  Evolutionary algorithms: from recombination to search distributions , 2001 .

[54]  Isabelle Bloch,et al.  Solving Graph Matching with EDAs Using a Permutation-Based Representation , 2002, Estimation of Distribution Algorithms.

[55]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[56]  Pedro Larrañaga,et al.  Mathematical Modeling of Discrete Estimation of Distribution Algorithms , 2002, Estimation of Distribution Algorithms.

[57]  Nicolas Monmarché,et al.  On the similarities between AS, BSC and PBIL: toward the birth of a new meta-heuristic , 1999 .

[58]  Fernando G. Lobo,et al.  Compressed introns in a linkage learning genetic algorithm , 1998 .

[59]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[60]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[61]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[62]  Heinz Mühlenbein,et al.  A Factorized Distribution Algorithm Using Single Connected Bayesian Networks , 2000, PPSN.

[63]  David E. Goldberg,et al.  Hierarchical Problem Solving and the Bayesian Optimization Algorithm , 2000, GECCO.

[64]  Ernesto Costa,et al.  Probabilistic Evolution and the Busy Beaver Problem , 2000, GECCO.

[65]  Gang Wang,et al.  Revisiting the GEMGA: scalable evolutionary optimization through linkage learning , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[66]  BayesiannetworksPedro,et al.  Combinatorial optimization by learning and simulation of , 2000 .

[67]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[68]  Rich Caruana,et al.  Removing the Genetics from the Standard Genetic Algorithm , 1995, ICML.

[69]  Byoung-Tak Zhang,et al.  Bayesian Evolutionary Optimization Using Helmholtz Machines , 2000, PPSN.

[70]  C.H.M. vanKemenade.  Building block filtering and mixing , 1998 .

[71]  S. Baluja.  An Empirical Comparison of Seven Iterative and Evolutionary Function Optimization Heuristics , 1995 .

[72]  Steffen L. Lauritzen,et al.  Graphical models in R , 1996 .

[73]  H. Mühlenbein,et al.  Gene Pool Recombination in Genetic Algorithms , 1996 .

[74]  M. Pelikán,et al.  The Bivariate Marginal Distribution Algorithm , 1999 .

[75]  Kalyanmoy Deb,et al.  RapidAccurate Optimization of Difficult Problems Using Fast Messy Genetic Algorithms , 1993, ICGA.

[76]  Pedro Larrañaga,et al.  Benefits of Data Clustering in Multimodal Function Optimization via EDAs , 2002, Estimation of Distribution Algorithms.

[77]  Markus H ohfeld,et al.  Random keys genetic algorithm with adaptive penalty function for optimization of constrained facility layout problems , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[78]  J. A. Lozano,et al.  Analyzing the PBIL Algorithm by Means of Discrete Dynamical Systems , 2000 .

[79]  Mandayam A. L. Thathachar,et al.  Learning Optimal Discriminant Functions through a Cooperative Game of Automata , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[80]  Colin Fyfe.  Structured population-based incremental learning , 1998, Soft Comput..

[81]  P. Larra,et al.  Feature Subset Selection by Bayesian Networks Based Optimization Abstract|a New Method for Feature Subset Selection in Machine Learning, Fss-ebna , 1999 .

[82]  Gilbert Syswerda,et al.  Simulated Crossover in Genetic Algorithms , 1992, FOGA.

[83]  Arnaud Berny.  Selection and Reinforcement Learning for Combinatorial Optimization , 2000, PPSN.