Bivariate estimation-of-distribution algorithms can find an exponential number of optima

Finding a large set of optima in a multimodal optimization landscape is a challenging task. Classical population-based evolutionary algorithms (EAs) typically converge only to a single solution. While this can be counteracted by applying niching strategies, the number of optima is nonetheless trivially bounded by the population size. Estimation-of-distribution algorithms (EDAs) are an alternative, maintaining a probabilistic model of the solution space instead of an explicit population. Such a model is able to implicitly represent a solution set that is far larger than any realistic population size. To support the study of how optimization algorithms handle large sets of optima, we propose the test function EqalBlocksOneMax (EBOM). It has an easy to optimize fitness landscape, however, with an exponential number of optima. We show that the bivariate EDA mutual-information-maximizing input clustering (MIMIC), without any problem-specific modification, quickly generates a model that behaves very similarly to a theoretically ideal model for that function, which samples each of the exponentially many optima with the same maximal probability.

[1]  Kalyanmoy Deb,et al.  Comparison of multi-modal optimization algorithms based on evolutionary algorithms , 2006, GECCO.

[2]  Wen-Lian Hsu,et al.  Multivariate multi-model approach for globally multimodal problems , 2010, GECCO '10.

[3]  Martin Pelikan,et al.  Analyzing probabilistic models in hierarchical BOA on traps and spin glasses , 2007, GECCO '07.

[4]  Benjamin Doerr,et al.  The Univariate Marginal Distribution Algorithm Copes Well with Deception and Epistasis , 2020, Evolutionary Computation.

[5]  Petr Posík.  Estimation of Distribution Algorithms , 2006 .

[6]  Xavier Llorà,et al.  Evolutionary computation and multimodal search: A good combination to tackle molecular diversity in the field of peptide design , 2007, Molecular Diversity.

[7]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[8]  D. Goldberg,et al.  Escaping hierarchical traps with competent genetic algorithms , 2001 .

[9]  Paul A. Viola,et al.  MIMIC: Finding Optima by Estimating Probability Densities , 1996, NIPS.

[10]  Arthur C. Sanderson,et al.  Multimodal Function Optimization Using Minimal Representation Size Clustering and Its Application to Planning Multipaths , 1997, Evolutionary Computation.

[11]  David E. Goldberg,et al.  Hierarchical BOA Solves Ising Spin Glasses and MAXSAT , 2003, GECCO.

[12]  Carsten Witt,et al.  Theory of estimation-of-distribution algorithms , 2018, GECCO.

[13]  Samir W. Mahfoud.  Niching methods for genetic algorithms , 1996 .

[14]  Michael J. Shaw,et al.  Genetic algorithms with dynamic niche sharing for multimodal function optimization , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[15]  M. Pelikán,et al.  The Bivariate Marginal Distribution Algorithm , 1999 .

[16]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[17]  Tobias Friedrich,et al.  EDAs cannot be Balanced and Stable , 2016, GECCO.

[18]  Pedro Larrañaga,et al.  Globally Multimodal Problem Optimization Via an Estimation of Distribution Algorithm Based on Unsupervised Learning of Bayesian Networks , 2005, Evolutionary Computation.

[19]  Per Kristian Lehre,et al.  On the limitations of the univariate marginal distribution algorithm to deception and where bivariate EDAs might help , 2019, FOGA '19.

[20]  Weijie Zheng,et al.  Sharp Bounds for Genetic Drift in EDAs , 2019, ArXiv.

[21]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[22]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[23]  Kumara Sastry,et al.  Linkage Learning via Probabilistic Modeling in the Extended Compact Genetic Algorithm (ECGA) , 2006, Scalable Optimization via Probabilistic Modeling.