Parameter Control of Genetic Algorithms by Learning and Simulation of Bayesian Networks — A Case Study for the Optimal Ordering of Tables

Parameter setting for evolutionary algorithms is still an important issue in evolutionary computation. There are two main approaches to parameter setting: parameter tuning and parameter control. In this paper, we introduce self-adaptive parameter control of a genetic algorithm based on Bayesian network learning and simulation. The nodes of this Bayesian network are genetic algorithm parameters to be controlled. Its structure captures probabilistic conditional (in)dependence relationships between the parameters. They are learned from the best individuals, i.e., the best configurations of the genetic algorithm. Individuals are evaluated by running the genetic algorithm for the respective parameter configuration. Since all these runs are time-consuming tasks, each genetic algorithm uses a small-sized population and is stopped before convergence. In this way promising individuals should not be lost. Experiments with an optimal search problem for simultaneous row and column orderings yield the same optima as state-of-the-art methods but with a sharp reduction in computational time. Moreover, our approach can cope with as yet unsolved high-dimensional problems.

[1]  Ritu Gupta,et al.  Statistical exploratory analysis of genetic algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[2]  Stefan Niermann Optimizing the Ordering of Tables With Evolutionary Computation , 2005 .

[3]  Martin Pelikan,et al.  Bayesian Optimization Algorithm , 2005 .

[4]  Zbigniew Michalewicz,et al.  Parameter Control in Evolutionary Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[5]  Marcus Gallagher,et al.  Combining Meta-EAs and Racing for Difficult EA Parameter Tuning Tasks , 2007, Parameter Setting in Evolutionary Algorithms.

[6]  A. E. Eiben,et al.  Costs and Benefits of Tuning Parameters of Evolutionary Algorithms , 2008, PPSN.

[7]  Héctor Pomares,et al.  Statistical analysis of the main parameters involved in the design of a genetic algorithm , 2002, IEEE Trans. Syst. Man Cybern. Part C.

[8]  Qingfu Zhang,et al.  DE/EDA: A new evolutionary algorithm for global optimization , 2005, Inf. Sci..

[9]  Xin Yao,et al.  NichingEDA: Utilizing the diversity inside a population of EDAs for continuous optimization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[10]  Jacques Bertin,et al.  Graphics and graphic information-processing , 1981 .

[11]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[12]  Berwin A. Turlach,et al.  Statistical exploratory analysis of genetic algorithms: the importance of interaction , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[13]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[14]  Gisele L. Pappa,et al.  Tuning Genetic Programming parameters with factorial designs , 2010, IEEE Congress on Evolutionary Computation.

[15]  Pedro Larrañaga,et al.  GA-EDA: hybrid evolutionary algorithm using genetic and estimation of distribution algorithms , 2004 .

[16]  Pedro Larrañaga,et al.  Combining variable neighborhood search and estimation of distribution algorithms in the protein side chain placement problem , 2007, J. Heuristics.

[17]  Pedro Larrañaga,et al.  Genetic Algorithms for the Travelling Salesman Problem: A Review of Representations and Operators , 1999, Artificial Intelligence Review.

[18]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[19]  Zbigniew Michalewicz,et al.  Adaptation in evolutionary computation: a survey , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[20]  Jim E. Smith Self adaptation in evolutionary algorithms , 1998 .

[21]  Hans-Georg Beyer,et al.  Self-Adaptation in Evolutionary Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[22]  Pedro Larrañaga,et al.  Adaptive Estimation of Distribution Algorithms , 2008, Adaptive and Multilevel Metaheuristics.

[23]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[24]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[25]  A. E. Eiben,et al.  Parameter Tuning of Evolutionary Algorithms: Generalist vs. Specialist , 2010, EvoApplications.

[26]  Ingo Rechenberg,et al.  Evolutionsstrategie '94 , 1994, Werkstatt Bionik und Evolutionstechnik.

[27]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[28]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[29]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[30]  Christos H. Papadimitriou,et al.  Computational complexity , 1993 .

[31]  Marimuthu Palaniswami,et al.  Computational Intelligence: A Dynamic System Perspective , 1995 .

[32]  B. Freisleben,et al.  Optimization of Genetic Algorithms by Genetic Algorithms , 1993 .

[33]  Peter J. Angeline,et al.  Adaptive and Self-adaptive Evolutionary Computations , 1995 .

[34]  G. Croes A Method for Solving Traveling-Salesman Problems , 1958 .

[35]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[36]  Cláudio F. Lima,et al.  Adaptive Population Sizing Schemes in Genetic Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[37]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[38]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[39]  Concha Bielza,et al.  Multidimensional statistical analysis of the parameterization of a genetic algorithm for the optimal ordering of tables , 2010, Expert Syst. Appl..

[40]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[41]  Oliver Kramer Self-Adaptive Heuristics for Evolutionary Computation , 2008, Studies in Computational Intelligence.

[42]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[43]  Nir Friedman,et al.  On the Sample Complexity of Learning Bayesian Networks , 1996, UAI.