Symmetry breaking in population-based optimization

Argues that the performance of evolutionary algorithms working on hard optimization problems depends strongly on how the population breaks the "symmetry" of the search space. The splitting of the search space into widely separate regions containing local optima is a generic property of a large class of hard optimization problem. This phenomenon is discussed by reference to two well studied examples, the Ising perceptron and the satisfiability problem (K-SAT). A finite population will quickly concentrate on one region of the search space. The cost of crossover between solutions in different regions of search space can accelerate this symmetry breaking. This, in turn, can dramatically reduce the amount of exploration, leading to suboptimal solutions being found. An analysis of symmetry breaking using diffusion model techniques borrowed from classical population genetics is presented. This shows how symmetry breaking depends on parameters such as the population size and selection rate.

[1]  R. Durrett.  Probability Models for DNA Sequence Evolution , 2002 .

[2]  E. Gardner,et al.  Optimal storage properties of neural network models , 1988 .

[3]  Rémi Monasson,et al.  Determining computational complexity from characteristic ‘phase transitions’ , 1999, Nature.

[4]  Jin-Kao Hao,et al.  Hybrid Evolutionary Algorithms for Graph Coloring , 1999, J. Comb. Optim..

[5]  Ingo Wegener,et al.  Real royal road functions--where crossover provably is essential , 2001, Discret. Appl. Math..

[6]  T. Ohta,et al.  The Average Number of Generations until Fixation of a Mutant Gene in a Finite Population. , 1969, Genetics.

[7]  H. Horner.  Dynamics of learning for the binary perceptron problem , 1992 .

[8]  Adam Prügel-Bennett,et al.  Barrier Trees For Search Analysis , 2003, GECCO.

[9]  A. Crisanti,et al.  The 3-SAT problem with large number of clauses in the ∞-replica symmetry breaking scheme , 2001, cond-mat/0108433.

[10]  Adam Prügel-Bennett,et al.  When a genetic algorithm outperforms hill-climbing , 2004, Theor. Comput. Sci..

[11]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[12]  Carlos Bustamante,et al.  Population Genetics of Molecular Evolution , 2005 .

[13]  Erick Cantú-Paz,et al.  Efficient and Accurate Parallel Genetic Algorithms , 2000, Genetic Algorithms and Evolutionary Computation.

[14]  W. Provine.  Sewall Wright and evolutionary biology , 1987 .

[15]  M. Mézard,et al.  Random K-satisfiability problem: from an analytic solution to an efficient algorithm. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[16]  W. Ewens.  Mathematical Population Genetics , 1980 .

[17]  Adam Pr.  Finite Population Effects for Ranking and Tournament Selection , 2000 .

[18]  N. Barton,et al.  PERSPECTIVE: A CRITIQUE OF SEWALL WRIGHT'S SHIFTING BALANCE THEORY OF EVOLUTION , 1997, Evolution; international journal of organic evolution.

[19]  R. A. Fisher,et al.  On the dominance ratio , 1990 .

[20]  E. Gardner.  The space of interactions in neural network models , 1988 .

[21]  Nicholas J. Radcliffe,et al.  Genetic Set Recombination , 1992, FOGA.

[22]  Michael T. Wolfinger,et al.  Barrier Trees of Degenerate Landscapes , 2002 .

[23]  W. Krauth,et al.  Storage capacity of memory networks with binary couplings , 1989 .

[24]  Rory A. Fisher,et al.  XVII—The distribution of gene ratios for rare mutations , 1931 .

[25]  M. Kimura,et al.  An introduction to population genetics theory , 1971 .

[26]  Rémi Monasson,et al.  Statistical mechanics methods and phase transitions in optimization problems , 2001, Theor. Comput. Sci..

[27]  Celia A. Glass,et al.  Genetic Algorithm for Graph Coloring: Exploration of Galinier and Hao's Algorithm , 2003, J. Comb. Optim..

[28]  Adam Prügel-Bennett,et al.  Genetic algorithm dynamics in two-well potentials with basins and barrier , 1997 .

[29]  A. Prügel-Bennett,et al.  The dynamics of a genetic algorithm for simple random Ising systems , 1997 .

[30]  Ingo Wegener,et al.  On the Analysis of Evolutionary Algorithms , 2001 .

[31]  Christian M. Reidys,et al.  Combinatorial Landscapes , 2002, SIAM Rev..

[32]  S. Wright,et al.  Evolution in Mendelian Populations. , 1931, Genetics.

[33]  M. Kimura.  Difiusion models in population genetics , 1964 .

[34]  M. Mézard,et al.  Analytic and Algorithmic Solution of Random Satisfiability Problems , 2002, Science.

[35]  P. Stadler.  Spectral Landscape Theory , 1999 .

[36]  B. Øksendal.  Stochastic Differential Equations , 1985 .

[37]  Adam Prügel-Bennett,et al.  The Dynamics of a Genetic Algorithm on a Model Hard Optimization Problem , 2000, Complex Syst..

[38]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[39]  A. Rogers,et al.  A solvable model of a hard optimisation problem , 2001 .