Sporadic Model Building for Efficiency Enhancement of hBOA Martin Pelikan

This paper describes and analyzes sporadic model building, which can be used to enhance the efficiency of the hierarchical Bayesian optimization algorithm (hBOA) and other estimation of distribution algorithms (EDAs). With sporadic model building, the structure of the probabilistic model is updated once every few iterations (generations), whereas in the remaining iterations only model parameters (conditional and marginal probabilities) are updated. Since the time complexity of updating model parameters is much lower than the time complexity of learning the model structure, sporadic model building decreases the overall time complexity of model building. The paper shows that for boundedly difficult nearly decomposable and hierarchical optimization problems, sporadic model building leads to a significant model-building speedup that decreases the asymptotic time complexity of model building in hBOA by a factor of Θ(n) to Θ(n), where n is the problem size. On the other hand, sporadic model building also increases the number of evaluations until convergence; nonetheless, for decomposable problems, the evaluation slowdown is insignificant compared to the gains in the asymptotic complexity of model building.

[1]  Ronald A. Howard,et al.  Influence Diagrams , 2005, Decis. Anal..

[2]  David E. Goldberg,et al.  Efficient Genetic Algorithms Using Discretization Scheduling , 2005, Evolutionary Computation.

[3]  Kalyanmoy Deb,et al.  Sufficient conditions for deceptive and easy binary functions , 1994, Annals of Mathematics and Artificial Intelligence.

[4]  David E. Goldberg,et al.  Efficiency enhancement of genetic algorithms via building-block-wise fitness estimation , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[5]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[6]  Franz Rothlauf,et al.  Evaluation-Relaxation Schemes for Genetic and Evolutionary Algorithms , 2004 .

[7]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[8]  David E. Goldberg,et al.  A hierarchy machine: Learning to optimize from nature and humans , 2003, Complex..

[9]  David E. Goldberg,et al.  Scalability of the Bayesian optimization algorithm , 2002, Int. J. Approx. Reason..

[10]  David E. Goldberg,et al.  Bayesian Optimization Algorithm: From Single Level to Hierarchy , 2002 .

[11]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[12]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[13]  D. Goldberg,et al.  Escaping hierarchical traps with competent genetic algorithms , 2001 .

[14]  D. Goldberg,et al.  Verification and extension of the theory of global-local hybrids , 2001 .

[15]  Fernando G. Lobo,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[16]  Dirk Thierens,et al.  Linkage Information Processing In Distribution Estimation Algorithms , 1999, GECCO.

[17]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[18]  E. Cantu-Paz,et al.  The Gambler's Ruin Problem, Genetic Algorithms, and the Sizing of Populations , 1997, Evolutionary Computation.

[19]  G. Harik.  Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[20]  Jordan B. Pollack,et al.  Modeling Building-Block Interdependency , 1998, PPSN.

[21]  D. Goldberg,et al.  Domino convergence, drift, and the temporal-salience structure of problems , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[22]  David Maxwell Chickering,et al.  A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.

[23]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[24]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[25]  Georges R. Harik,et al.  Finding Multimodal Solutions Using Restricted Tournament Selection , 1995, ICGA.

[26]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[27]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[28]  Kalyanmoy Deb,et al.  Genetic Algorithms, Noise, and the Sizing of Populations , 1992, Complex Syst..

[29]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[30]  Ronald A. Howard,et al.  Readings on the Principles and Applications of Decision Analysis , 1989 .

[31]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[32]  D. E. Goldberg,et al.  Simple Genetic Algorithms and the Minimal, Deceptive Problem , 1987 .

[33]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[34]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[35]  David M. Miller,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[36]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .