On the Taxonomy of Optimization Problems Under Estimation of Distribution Algorithms

Understanding the relationship between a search algorithm and the space of problems is a fundamental issue in the optimization field. In this paper, we lay the foundations to elaborate taxonomies of problems under estimation of distribution algorithms (EDAs). By using an infinite population model and assuming that the selection operator is based on the rank of the solutions, we group optimization problems according to the behavior of the EDA. Throughout the definition of an equivalence relation between functions it is possible to partition the space of problems in equivalence classes in which the algorithm has the same behavior. We show that only the probabilistic model is able to generate different partitions of the set of possible problems and hence, it predetermines the number of different behaviors that the algorithm can exhibit. As a natural consequence of our definitions, all the objective functions are in the same equivalence class when the algorithm does not impose restrictions to the probabilistic model. The taxonomy of problems, which is also valid for finite populations, is studied in depth for a simple EDA that considers independence among the variables of the problem. We provide the sufficient and necessary condition to decide the equivalence between functions and then we develop the operators to describe and count the members of a class. In addition, we show the intrinsic relation between univariate EDAs and the neighborhood system induced by the Hamming distance by proving that all the functions in the same class have the same number of local optima and that they are in the same ranking positions. Finally, we carry out numerical simulations in order to analyze the different behaviors that the algorithm can exhibit for the functions defined over the search space .

[1]  Martin Pelikan,et al.  An application of a multivariate estimation of distribution algorithm to cancer chemotherapy , 2008, GECCO '08.

[2]  J. A. Lozano,et al.  Analyzing the PBIL Algorithm by Means of Discrete Dynamical Systems , 2000 .

[3]  Jason H. Moore,et al.  Filling the gap between biology and computer science , 2008, BioData Mining.

[4]  Martin Pelikan,et al.  Analyzing Probabilistic Models in Hierarchical BOA , 2009, IEEE Transactions on Evolutionary Computation.

[5]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[6]  Michael D. Vose,et al.  The simple genetic algorithm - foundations and theory , 1999, Complex adaptive systems.

[7]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[8]  Hussein A. Abbass,et al.  Motif Difficulty (MD): A Predictive Measure of Problem Difficulty for Evolutionary Algorithms Using Network Motifs , 2012, Evolutionary Computation.

[9]  Pedro Larrañaga,et al.  Protein Folding in Simplified Models With Estimation of Distribution Algorithms , 2008, IEEE Transactions on Evolutionary Computation.

[10]  Pedro Larrañaga,et al.  Mathematical modelling of UMDAc algorithm with tournament selection. Behaviour on linear and quadratic functions , 2002, Int. J. Approx. Reason..

[11]  David E. Goldberg,et al.  Using Previous Models to Bias Structural Learning in the Hierarchical BOA , 2012, Evolutionary Computation.

[12]  Lothar Thiele,et al.  A Comparison of Selection Schemes Used in Evolutionary Algorithms , 1996, Evolutionary Computation.

[13]  Bart Naudts,et al.  A comparison of predictive measures of problem difficulty in evolutionary algorithms , 2000, IEEE Trans. Evol. Comput..

[14]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[15]  Benjamin Doerr,et al.  Ranking-Based Black-Box Complexity , 2011, Algorithmica.

[16]  Mohamed A. El-Sharkawi,et al.  Modern Heuristic Optimization Techniques , 2008 .

[17]  Qingfu Zhang,et al.  On the limits of effectiveness in estimation of distribution algorithms , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[18]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[19]  Concha Bielza,et al.  A review of estimation of distribution algorithms in bioinformatics , 2008, BioData Mining.

[20]  Adam Pr.  Finite Population Effects for Ranking and Tournament Selection , 2000 .

[21]  Xin Yao,et al.  Analysis of Computational Time of Simple Estimation of Distribution Algorithms , 2010, IEEE Transactions on Evolutionary Computation.

[22]  Qingfu Zhang,et al.  On stability of fixed points of limit models of univariate marginal distribution algorithm and factorized distribution algorithm , 2004, IEEE Transactions on Evolutionary Computation.

[23]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[24]  Peter A. N. Bosman,et al.  The anticipated mean shift and cluster registration in mixture-based EDAs for multi-objective optimization , 2010, GECCO '10.

[25]  Mohamed A. El-Sharkawi,et al.  Modern heuristic optimization techniques :: theory and applications to power systems , 2008 .

[26]  Thomas Jansen,et al.  Design and Management of Complex Technical Processes and Systems by Means of Computational Intelligence Methods on Classifications of Fitness Functions on Classifications of Fitness Functions , 2022 .

[27]  Per Kristian Lehre,et al.  Black-Box Search by Unbiased Variation , 2010, GECCO '10.

[28]  Pedro Larrañaga,et al.  Research topics in discrete estimation of distribution algorithms based on factorizations , 2009, Memetic Comput..

[29]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[30]  Edward R. Scheinerman,et al.  Invitation to Dynamical Systems , 1995 .

[31]  Qingfu Zhang,et al.  On the convergence of a class of estimation of distribution algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[32]  Jonathan L. Shapiro,et al.  Drift and Scaling in Estimation of Distribution Algorithms , 2005, Evolutionary Computation.

[33]  Heinz Mühlenbein,et al.  FDA -A Scalable Evolutionary Algorithm for the Optimization of Additively Decomposed Functions , 1999, Evolutionary Computation.

[34]  Roberto Santana,et al.  Toward Understanding EDAs Based on Bayesian Networks Through a Quantitative Analysis , 2012, IEEE Transactions on Evolutionary Computation.

[35]  Yong Gao,et al.  Space Complexity of Estimation of Distribution Algorithms , 2005, Evolutionary Computation.

[36]  Heinz Mühlenbein,et al.  Schemata, Distributions and Graphical Models in Evolutionary Optimization , 1999, J. Heuristics.