Optimization with extremal dynamics

A local-search heuristic for finding high-quality solutions for many hard optimization problems is explored. The method is inspired by recent progress in understanding far-from-equilibrium phenomena in terms of self-organized criticality, a concept introduced to describe emergent complexity in physical systems. This method, called extremal optimization, successively replaces the value of extremely undesirable variables in a sub-optimal solution with new, random ones. Large, avalanche-like fluctuations in the cost function emerge dynamically. These enable the search to effectively scaling barriers to explore local optima in distant neighborhoods of the configuration space while eliminating the need to tune parameters. Drawing upon models used to simulate the dynamics of granular media, evolution, or geology, extremal optimization complements approximation methods inspired by equilibrium statistical physics, such as simulated annealing. This method is very general and so far has proved competitive with--and even superior to--more elaborate general-purpose heuristics on testbeds of constrained optimization problems with up to 105 variables, such as bipartitioning, coloring, and spin glasses. Analysis of a model problem predicts the only free parameter of the method in accordance with all experimental results.

[1]  A. Percus,et al.  Nature's Way of Optimizing , 1999, Artif. Intell..

[2]  P. Bak,et al.  Self-organized criticality. , 1988, Physical review. A, General physics.

[3]  Paczuski,et al.  Avalanche dynamics in evolution, growth, and depinning models. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[4]  M. Mézard,et al.  The Cavity Method at Zero Temperature , 2002, cond-mat/0207121.

[5]  Tang,et al.  Self-Organized Criticality: An Explanation of 1/f Noise , 2011 .

[6]  Stefan Boettcher,et al.  Jamming Model for the Extremal Optimization Heuristic , 2001, ArXiv.

[7]  A K Hartmann.  Calculation of ground states of four-dimensional +/-J Ising spin glasses. , 1999, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[8]  S. Boettcher.  Numerical results for ground states of mean-field spin glasses at low connectivities , 2002, cond-mat/0208444.

[9]  Paczuski,et al.  Exact results for spatiotemporal correlations in a self-organized critical model of punctuated equilibrium. , 1995, Physical review letters.

[10]  Alexander K. Hartmann.  GROUND-STATE BEHAVIOR OF THE THREE-DIMENSIONAL J RANDOM-BOND ISING MODEL , 1999 .

[11]  Tamás Vicsek,et al.  Power-law distribution of landslides in an experiment on the erosion of a granular pile , 1994 .

[12]  S. Boettcher.  Extremal Optimization of Graph Partitioning at the Percolation Threshold , 1999, cond-mat/9901353.

[13]  Cecilia R. Aragon,et al.  Optimization by Simulated Annealing: An Experimental Evaluation; Part I, Graph Partitioning , 1989, Oper. Res..

[14]  Károly F. Pál,et al.  The ground state energy of the Edwards-Anderson Ising spin glass with a hybrid genetic algorithm , 1996 .

[15]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[16]  Per Bak,et al.  How Nature Works , 1996 .

[17]  A. Maritan,et al.  Models of Fractal River Basins , 1998, cond-mat/9803287.

[18]  Joseph C. Culberson,et al.  Frozen development in graph coloring , 2001, Theor. Comput. Sci..

[19]  F. Barahona.  On the computational complexity of Ising spin glass models , 1982 .

[20]  M. Mézard,et al.  Spin Glass Theory and Beyond , 1987 .

[21]  Alexander K. Hartmann,et al.  Evidence for nontrivial ground-state structure of 3d ? J spin glasses , 1997 .

[22]  Bak,et al.  Punctuated equilibrium and criticality in a simple model of evolution. , 1993, Physical review letters.

[23]  Peter C. Cheeseman,et al.  Where the Really Hard Problems Are , 1991, IJCAI.

[24]  Tad Hogg,et al.  Phase Transitions and the Search Problem , 1996, Artif. Intell..

[25]  Béla Bollobás,et al.  Random Graphs , 1985 .

[26]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[27]  Bruce Hendrickson,et al.  A Multi-Level Algorithm For Partitioning Graphs , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[28]  Stefan Boettcher,et al.  Extremal Optimization for Graph Partitioning , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[29]  T. H. Klotz,et al.  "Valley structures" in the phase space of a finite 3d ISING spin glass with , 1994 .

[30]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[31]  Bernd Freisleben,et al.  Memetic Algorithms and the Fitness Landscape of the Graph Bi-Partitioning Problem , 1998, PPSN.