Enhancing Efficiency of Hierarchical BOA Via Distance-Based Model Restrictions

This paper analyzes the effects of restricting probabilistic models in the hierarchical Bayesian optimization algorithm (hBOA) by defining a distance metric over variables and disallowing dependencies between variables at distances greater than a given threshold. We argue that by using prior problem-specific knowledge, it is often possible to develop a distance metric that closely corresponds to the strength of interactions between variables. This distance metric can then be used to speed up model building in hBOA. Three test problems are considered: 3D Ising spin glasses, random additively decomposable problems, and the minimum vertex cover.

[1]  Shumeet Baluja,et al.  Incorporating a priori Knowledge in Probabilistic-Model Based Optimization , 2006, Scalable Optimization via Probabilistic Modeling.

[2]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[3]  Béla Bollobás,et al.  Random Graphs: Notation , 2001 .

[4]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[5]  Martin V. Butz,et al.  Hierarchical BOA on random decomposable problems , 2006, GECCO '06.

[6]  David E. Goldberg,et al.  Scalability of the Bayesian optimization algorithm , 2002, Int. J. Approx. Reason..

[7]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[8]  F. Barahona.  On the computational complexity of Ising spin glass models , 1982 .

[9]  D. Goldberg,et al.  Don't evaluate, inherit , 2001 .

[10]  Martin V. Butz,et al.  Substructural Neighborhoods for Local Search in the Bayesian Optimization Algorithm , 2006, PPSN.

[11]  David E. Goldberg,et al.  Efficiency Enhancement of Estimation of Distribution Algorithms , 2006, Scalable Optimization via Probabilistic Modeling.

[12]  M. Weigt,et al.  Minimal vertex covers on finite-connectivity random graphs: a hard-sphere lattice-gas picture. , 2000, Physical review. E, Statistical, nonlinear, and soft matter physics.

[13]  Martin V. Butz,et al.  Performance of Evolutionary Algorithms on Random Decomposable Problems , 2006, PPSN.

[14]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[15]  J. A. Lozano,et al.  Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms (Studies in Fuzziness and Soft Computing) , 2006 .

[16]  Ronald A. Howard,et al.  Readings on the Principles and Applications of Decision Analysis , 1989 .

[17]  David E. Goldberg,et al.  Empirical analysis of ideal recombination on random decomposable problems , 2007, GECCO '07.

[18]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[19]  Heinz Mühlenbein,et al.  Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning , 2002, Int. J. Approx. Reason..

[20]  Franz Rothlauf,et al.  Evaluation-Relaxation Schemes for Genetic and Evolutionary Algorithms , 2004 .

[21]  Martin Pelikan,et al.  Hybrid evolutionary algorithms on minimum vertex cover for random graphs , 2007, GECCO '07.

[22]  Martin Pelikan,et al.  Fitness Inheritance in the Bayesian Optimization Algorithm , 2004, GECCO.

[23]  David E. Goldberg,et al.  Sporadic model building for efficiency enhancement of hierarchical BOA , 2006, GECCO '06.

[24]  Pedro Larrañaga,et al.  The Role of a Priori Information in the Minimization of Contact Potentials by Means of Estimation of Distribution Algorithms , 2007, EvoBIO.

[25]  Georges R. Harik,et al.  Finding Multimodal Solutions Using Restricted Tournament Selection , 1995, ICGA.

[26]  David Maxwell Chickering,et al.  A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.

[27]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[28]  Béla Bollobás,et al.  Random Graphs , 1985 .

[29]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[30]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[31]  Martin Pelikan,et al.  Analyzing probabilistic models in hierarchical BOA on traps and spin glasses , 2007, GECCO '07.

[32]  D. Goldberg,et al.  Escaping hierarchical traps with competent genetic algorithms , 2001 .

[33]  Martin Pelikan,et al.  Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications (Studies in Computational Intelligence) , 2006 .

[34]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.