From mating pool distributions to model overfitting

This paper addresses selection as a source of overfitting in Bayesian estimation of distribution algorithms (EDAs). The purpose of the paper is twofold. First, it shows how the selection operator can lead to model overfitting in the Bayesian optimization algorithm (BOA). Second, the metric score that guides the search for an adequate model structure is modified to take into account the non-uniform distribution of the mating pool generated by tournament selection.

[1]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[2]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[3]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[4]  Kalyanmoy Deb,et al.  Analyzing Deception in Trap Functions , 1992, FOGA.

[5]  David E. Goldberg,et al.  Influence of selection and replacement strategies on linkage learning in BOA , 2007, 2007 IEEE Congress on Evolutionary Computation.

[6]  Lothar Thiele,et al.  A Comparison of Selection Schemes Used in Evolutionary Algorithms , 1996, Evolutionary Computation.

[7]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[8]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[9]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[10]  Lothar Thiele,et al.  A Comparison of Selection Schemes used in Genetic Algorithms , 1995 .

[11]  Anne Brindle,et al.  Genetic algorithms for function optimization , 1980 .

[12]  Dirk Thierens,et al.  Mixing in Genetic Algorithms , 1993, ICGA.

[13]  David Maxwell Chickering,et al.  A Bayesian Approach to Learning Bayesian Networks with Local Structure , 1997, UAI.

[14]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[15]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[16]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[17]  Franz Rothlauf,et al.  Evaluation-Relaxation Schemes for Genetic and Evolutionary Algorithms , 2004 .

[18]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[19]  D. Ackley.  A connectionist machine for genetic hillclimbing , 1987 .