Research topics in discrete estimation of distribution algorithms based on factorizations

In this paper, we identify a number of topics relevant for the improvement and development of discrete estimation of distribution algorithms. Focusing on the role of probability distributions and factorizations in estimation of distribution algorithms, we present a survey of current challenges where further research must provide answers that extend the potential and applicability of these algorithms. In each case we state the research topic and elaborate on the reasons that make it relevant for estimation of distribution algorithms. In some cases current work or possible alternatives for the solution of the problem are discussed.

[1]  Pedro Larrañaga,et al.  Parallel Estimation of Distribution Algorithms , 2002, Estimation of Distribution Algorithms.

[2]  Jose Miguel Puerta,et al.  EDNA: Estimation of Dependency Networks Algorithm , 2007, IWINAC.

[3]  Dirk Thierens,et al.  Multi-objective mixture-based iterated density estimation evolutionary algorithms , 2001 .

[4]  David E. Goldberg,et al.  Population sizing for entropy-based model building in discrete estimation of distribution algorithms , 2007, GECCO '07.

[5]  Xiao Wang,et al.  Evolutionary optimization with Markov random field prior , 2004, IEEE Trans. Evol. Comput..

[6]  Siddhartha Shakya,et al.  Using a Markov network model in a univariate EDA: an empirical cost-benefit analysis , 2005, GECCO '05.

[7]  Zengqi Sun,et al.  Detecting the epistatic structure of generalized embedded landscape , 2007, Genetic Programming and Evolvable Machines.

[8]  David J. Hand,et al.  Graphical Models in Applied Multivariate Statistics. , 1990 .

[9]  David E. Goldberg,et al.  Hierarchical BOA Solves Ising Spin Glasses and MAXSAT , 2003, GECCO.

[10]  David E. Goldberg,et al.  Using Previous Models to Bias Structural Learning in the Hierarchical BOA , 2008, Evolutionary Computation.

[11]  Heinz Mühlenbein,et al.  Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning , 2002, Int. J. Approx. Reason..

[12]  Steffen L. Lauritzen,et al.  Graphical models in R , 1996 .

[13]  Raphael T. Haftka,et al.  A double-distribution statistical algorithm for composite laminate optimization , 2004 .

[14]  Jürgen Branke,et al.  A Multi-population Approach to Dynamic Optimization Problems , 2000 .

[15]  Siddhartha Shakya,et al.  Optimization by estimation of distribution with DEUM framework based on Markov random fields , 2007, Int. J. Autom. Comput..

[16]  Pedro Larrañaga,et al.  A Review on Estimation of Distribution Algorithms , 2002, Estimation of Distribution Algorithms.

[17]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[18]  Endika Bengoetxea,et al.  A parallel framework for loopy belief propagation , 2007, GECCO '07.

[19]  Qingfu Zhang,et al.  Modelling the Regularity in an Estimation of Distribution Algorithm for Continuous Multiobjective Optimization with Variable Linkages , 2006 .

[20]  Martin Pelikan,et al.  Analyzing probabilistic models in hierarchical BOA on traps and spin glasses , 2007, GECCO '07.

[21]  Masaharu Munetomo,et al.  Linkage Identification by Fitness Difference Clustering , 2006, Evolutionary Computation.

[22]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[23]  Dirk Thierens,et al.  Crossing the road to efficient IDEAs for permutation problems , 2001 .

[24]  Alden H. Wright,et al.  On the convergence of an estimation of distribution algorithm based on linkage discovery and factorization , 2005, GECCO '05.

[25]  Michele Leone,et al.  Clustering by Soft-constraint Affinity Propagation: Applications to Gene-expression Data , 2022 .

[26]  H. Mühlenbein.  Convergence of Estimation of Distribution Algorithms for Finite Samples , 2007 .

[27]  Thilo Mahnig,et al.  Comparing the adaptive Boltzmann selection schedule SDS to truncation selection , 2007 .

[28]  Jose Miguel Puerta,et al.  Migration of Probability Models Instead of Individuals: An Alternative When Applying the Island Model to EDAs , 2004, PPSN.

[29]  Andrew Johnson,et al.  The Importance of Selection Mechanisms in Distribution Estimation Algorithms , 2001, Artificial Evolution.

[30]  Pedro Larrañaga,et al.  Mixtures of Kikuchi Approximations , 2006, ECML.

[31]  Kumara Sastry,et al.  Linkage Learning via Probabilistic Modeling in the Extended Compact Genetic Algorithm (ECGA) , 2006, Scalable Optimization via Probabilistic Modeling.

[32]  ThieleLothar,et al.  A comparison of selection schemes used in evolutionary algorithms , 1996 .

[33]  Pedro Larrañaga,et al.  Solving the Traveling Salesman Problem with EDAs , 2002, Estimation of Distribution Algorithms.

[34]  G. Harik.  Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[35]  J. Ford,et al.  Hybrid estimation of distribution algorithm for global optimization , 2004 .

[36]  José Antonio Lozano,et al.  Scatter Search in software testing, comparison and collaboration with Estimation of Distribution Algorithms , 2006, Eur. J. Oper. Res..

[37]  Shumeet Baluja,et al.  Incorporating a priori Knowledge in Probabilistic-Model Based Optimization , 2006, Scalable Optimization via Probabilistic Modeling.

[38]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[39]  Pedro Larrañaga,et al.  Combinatonal Optimization by Learning and Simulation of Bayesian Networks , 2000, UAI.

[40]  William E. Hart,et al.  Recent Advances in Memetic Algorithms , 2008 .

[41]  Jiri Ocenasek,et al.  Estimation of Distribution Algorithm for Mixed Continuous-Discrete Optimization , 2002 .

[42]  Dirk Thierens,et al.  Numerical Optimization with Real-Valued Estimation-of-Distribution Algorithms , 2006, Scalable Optimization via Probabilistic Modeling.

[43]  Heinz Mühlenbein,et al.  The Estimation of Distributions and the Minimum Relative Entropy Principle , 2005, Evol. Comput..

[44]  Alberto Ochoa,et al.  Linking Entropy to Estimation of Distribution Algorithms , 2006, Towards a New Evolutionary Computation.

[45]  Pedro Larrañaga,et al.  Exact Bayesian network learning in estimation of distribution algorithms , 2007, 2007 IEEE Congress on Evolutionary Computation.

[46]  D. Nilsson,et al.  An efficient algorithm for finding the M most probable configurationsin probabilistic expert systems , 1998, Stat. Comput..

[47]  Martin Pelikan,et al.  Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications (Studies in Computational Intelligence) , 2006 .

[48]  Pedro Larrañaga,et al.  Interactions and dependencies in estimation of distribution algorithms , 2005, 2005 IEEE Congress on Evolutionary Computation.

[49]  Ernesto Costa,et al.  Too Busy to Learn , 2000 .

[50]  Mauro Birattari,et al.  Model-Based Search for Combinatorial Optimization: A Critical Survey , 2004, Ann. Oper. Res..

[51]  Griffin Caprio,et al.  Parallel Metaheuristics , 2008, IEEE Distributed Systems Online.

[52]  Pedro Larrañaga,et al.  Evolutionary computation based on Bayesian classifiers , 2004 .

[53]  David E. Goldberg,et al.  Influence of selection and replacement strategies on linkage learning in BOA , 2007, 2007 IEEE Congress on Evolutionary Computation.

[54]  X. Jin.  Factor graphs and the Sum-Product Algorithm , 2002 .

[55]  L. Kallel,et al.  Theoretical Aspects of Evolutionary Computing , 2001, Natural Computing Series.

[56]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[57]  D.E. Goldberg,et al.  A genetic algorithm using linkage identification by nonlinearity check , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[58]  Pedro Larrañaga,et al.  Adaptive Estimation of Distribution Algorithms , 2008, Adaptive and Multilevel Metaheuristics.

[59]  Martin Pelikan,et al.  Enhancing Efficiency of Hierarchical BOA Via Distance-Based Model Restrictions , 2008, PPSN.

[60]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[61]  Pedro Larrañaga,et al.  Triangulation of Bayesian networks with recursive estimation of distribution algorithms , 2009, Int. J. Approx. Reason..

[62]  David E. Goldberg,et al.  Bayesian Optimization Algorithm, Population Sizing, and Time to Convergence , 2000, GECCO.

[63]  Jeff A. Bilmes,et al.  Dynamic Bayesian Multinets , 2000, UAI.

[64]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[65]  Qingfu Zhang,et al.  On the convergence of a factorized distribution algorithm with truncation selection , 2004, Complex..

[66]  Roberto Santana,et al.  Estimation of Distribution Algorithms with Kikuchi Approximations , 2005, Evolutionary Computation.

[67]  Xavier Llorà,et al.  Towards billion-bit optimization via a parallel estimation of distribution algorithm , 2007, GECCO '07.

[68]  Adam Prügel-Bennett,et al.  Symmetry breaking in population-based optimization , 2004, IEEE Transactions on Evolutionary Computation.

[69]  BayesiannetworksPedro,et al.  Combinatorial optimization by learning and simulation of , 2000 .

[70]  Hussein A. Abbass,et al.  A Survey of Probabilistic Model Building Genetic Programming , 2006, Scalable Optimization via Probabilistic Modeling.

[71]  D. Goldberg,et al.  Population Sizing for Entropy-based Model Building in Genetic Algorithms , 2006 .

[72]  Heinz Mühlenbein,et al.  A Maximum Entropy Approach to Sampling in EDA ? The Single Connected Case , 2003, CIARP.

[73]  Lothar Thiele,et al.  A Comparison of Selection Schemes Used in Evolutionary Algorithms , 1996, Evolutionary Computation.

[74]  Y. Weiss,et al.  Finding the M Most Probable Configurations using Loopy Belief Propagation , 2003, NIPS 2003.

[75]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[76]  David E. Goldberg,et al.  Sporadic model building for efficiency enhancement of hierarchical BOA , 2006, GECCO.

[77]  Pedro Larrañaga,et al.  Globally Multimodal Problem Optimization Via an Estimation of Distribution Algorithm Based on Unsupervised Learning of Bayesian Networks , 2005, Evolutionary Computation.

[78]  Martin Pelikan,et al.  Searching for Ground States of Ising Spin Glasses with Hierarchical BOA and Cluster Exact Approximation , 2006, Scalable Optimization via Probabilistic Modeling.

[79]  Robin Hons,et al.  Estimation of Distribution Algorithms and Minimum Relative Entropy , 2005 .

[80]  Martin Pelikan,et al.  Design of Parallel Estimation of Distribution Algorithms , 2006, Scalable Optimization via Probabilistic Modeling.

[81]  Hisashi Handa.  Estimation of Distribution Algorithms with Mutation , 2005, EvoCOP.

[82]  J. A. Lozano,et al.  Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms (Studies in Fuzziness and Soft Computing) , 2006 .

[83]  Martin Pelikan,et al.  Fitness Inheritance in the Bayesian Optimization Algorithm , 2004, GECCO.

[84]  Pedro Larrañaga,et al.  Learning Bayesian Networks In The Space Of Orderings With Estimation Of Distribution Algorithms , 2004, Int. J. Pattern Recognit. Artif. Intell..

[85]  Pedro Larrañaga,et al.  The Role of a Priori Information in the Minimization of Contact Potentials by Means of Estimation of Distribution Algorithms , 2007, EvoBIO.

[86]  William T. Freeman,et al.  Constructing free-energy approximations and generalized belief propagation algorithms , 2005, IEEE Transactions on Information Theory.

[87]  Pedro Larrañaga,et al.  Optimization by Max-Propagation Using Kikuchi Approximations , 2007 .

[88]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[89]  Qingfu Zhang,et al.  An evolutionary algorithm with guided mutation for the maximum clique problem , 2005, IEEE Transactions on Evolutionary Computation.

[90]  Worasait Suwannik,et al.  Solving one-billion-bit Noisy OneMax problem using Estimation Distribution Algorithm with Arithmetic Coding , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[91]  Bernabé Dorronsoro,et al.  Estimation of Distribution Algorithms , 2006 .

[92]  Zengqi Sun,et al.  Extended probe method for linkage discovery over high-cardinality alphabets , 2007, GECCO '07.

[93]  Roberto Santana,et al.  The Factorized Distribution Algorithm and The Junction Tree: A Learning Perspective , 2005 .

[94]  Pedro Larrañaga,et al.  Side chain placement using estimation of distribution algorithms , 2007, Artif. Intell. Medicine.

[95]  Dirk Thierens,et al.  Linkage Information Processing In Distribution Estimation Algorithms , 1999, GECCO.

[96]  Jiahai Wang,et al.  Genetic Particle Swarm Optimization Based on Estimation of Distribution , 2007, LSMS.

[97]  A. Ochoa,et al.  A factorized distribution algorithm based on polytrees , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[98]  David E. Goldberg,et al.  Efficiency enhancement of genetic algorithms via building-block-wise fitness estimation , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[99]  María S. Pérez-Hernández,et al.  GA-EDA: A New Hybrid Cooperative Search Evolutionary Algorithm , 2006, Towards a New Evolutionary Computation.

[100]  Yoshinori Uesaka,et al.  Foundations of real-world intelligence , 2001 .

[101]  Endika Bengoetxea,et al.  Inexact Graph Matching Using Estimation of Distribution Algorithms , 2002 .

[102]  Heinz Mühlenbein,et al.  Optimal Mutation Rate Using Bayesian Priors for Estimation of Distribution Algorithms , 2001, SAGA.

[103]  Martin Pelikan,et al.  Scalable Optimization via Probabilistic Modeling , 2006, Studies in Computational Intelligence.

[104]  Qingfu Zhang,et al.  Approaches to selection and their effect on fitness modelling in an Estimation of Distribution Algorithm , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[105]  Qingfu Zhang,et al.  On the convergence of a class of estimation of distribution algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[106]  Jonathan L. Shapiro,et al.  Drift and Scaling in Estimation of Distribution Algorithms , 2005, Evolutionary Computation.

[107]  T. Mahnig,et al.  Evolutionary algorithms: from recombination to search distributions , 2001 .

[108]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[109]  Martin J. Wainwright,et al.  MAP estimation via agreement on trees: message-passing and linear programming , 2005, IEEE Transactions on Information Theory.

[110]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[111]  Riccardo Zecchina,et al.  Survey propagation: An algorithm for satisfiability , 2002, Random Struct. Algorithms.

[112]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[113]  Pedro Larrañaga,et al.  Combining Bayesian classifiers and estimation of distribution algorithms for optimization in continuous domains , 2007, Connect. Sci..

[114]  Pedro Larrañaga,et al.  The Impact of Exact Probabilistic Learning Algorithms in EDAs Based on Bayesian Networks , 2008, Linkage in Evolutionary Computation.

[115]  Heinz Mühlenbein,et al.  Schemata, Distributions and Graphical Models in Evolutionary Optimization , 1999, J. Heuristics.

[116]  David E. Goldberg,et al.  Hierarchical Bayesian Optimization Algorithm , 2006, Scalable Optimization via Probabilistic Modeling.

[117]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[118]  David E. Goldberg,et al.  Efficiency Enhancement of Estimation of Distribution Algorithms , 2006, Scalable Optimization via Probabilistic Modeling.

[119]  Peter A. N. Bosman,et al.  Matching inductive search bias and problem structure in continuous Estimation-of-Distribution Algorithms , 2008, Eur. J. Oper. Res..

[120]  Martin J. Wainwright,et al.  MAP estimation via agreement on (hyper)trees: Message-passing and linear programming , 2005, ArXiv.

[121]  Ryszard S. Michalski,et al.  LEARNABLE EVOLUTION MODEL: Evolutionary Processes Guided by Machine Learning , 2004, Machine Learning.

[122]  J. Urgen Branke.  Evolutionary Approaches to Dynamic Optimization Problems -a Survey , 1999 .

[123]  Pedro Larrañaga,et al.  GA-EDA: hybrid evolutionary algorithm using genetic and estimation of distribution algorithms , 2004 .

[124]  Pedro Larrañaga,et al.  Combining variable neighborhood search and estimation of distribution algorithms in the protein side chain placement problem , 2007, J. Heuristics.

[125]  Julian F. Miller,et al.  Genetic and Evolutionary Computation — GECCO 2003 , 2003, Lecture Notes in Computer Science.

[126]  Alexander Mendiburu,et al.  Parallel implementation of EDAs based on probabilistic graphical models , 2005, IEEE Transactions on Evolutionary Computation.

[127]  Yair Weiss,et al.  Approximate Inference and Protein-Folding , 2002, NIPS.

[128]  Qingfu Zhang,et al.  This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 1 RM-MEDA: A Regularity Model-Based Multiobjective Estimation of , 2022 .

[129]  C. Reeves,et al.  Properties of fitness functions and search landscapes , 2001 .

[130]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[131]  Heinz Mühlenbein,et al.  The Factorized Distribution Algorithm and the Minimum Relative Entropy Principle , 2006, Scalable Optimization via Probabilistic Modeling.

[132]  Dirk Thierens,et al.  Multi-objective optimization with diversity preserving mixture-based iterated density estimation evolutionary algorithms , 2002, Int. J. Approx. Reason..

[133]  Enrique Alba,et al.  Parallel Estimation of Distribution Algorithms , 2005 .

[134]  Pedro Larrañaga,et al.  Protein Folding in Simplified Models With Estimation of Distribution Algorithms , 2008, IEEE Transactions on Evolutionary Computation.

[135]  Roberto Santana Hermida.  Advances in probabilistic graphical models for optimisation and learning. Applications in protein modeling , 2006 .

[136]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[137]  Alejandro Rosete-Suárez.  Efficient-Discarding Fitness Functions , 2007 .

[138]  A. Hartmann.  Phase Transitions in Combinatorial Optimization Problems - Basics, Algorithms and Statistical Mechanics , 2005 .