Theory and practice of simulated annealing on special energy landscapes

A new theoretical framework for analyzing simulated annealing is presented. The behavior of simulated annealing depends crucially on the "energy landscape" associated with the optimization problem: the landscape must have special properties if annealing is to be efficient. For a class of deterministic fractal landscapes, and for linearly separable functions, annealing is proved to be efficient, running in time polynomial- or quasi-polynomial-time in the solution quality. The cooling schedule employed is the familiar geometric schedule of annealing practice, rather than the logarithmic schedule of previous theory. While random sampling and descent algorithms may be more efficient than annealing in some cases, their run times increase exponentially with the number of dimensions, while annealing's increases only linearly. A number of circuit placement problems are found to have energy landscapes with weaker, random fractal properties, giving for the first time a reasonable explanation of the successful application of simulated annealing to problems in the VLSI domain. The analysis models annealing as a random walk on a graph, and uses recent theorems relating the "conductance" of a graph to the mixing rate of its associated Markov chain. This enables both a new conceptual approach and new quantitative methods. Another foundation of the analysis is a proof that the expected energy cannot increase during annealing with decreasing temperatures; a surprising "probability pump" counterexample exists if the hypotheses are weakened slightly. For the model energy landscapes, an optimal "hypergeometric" cooling schedule is derived, and its efficiency compared with that of geometric cooling schedules. Experiments on a "real-world" circuit partitioning problem demonstrate increased efficiency close to the predicted amount. Other model-based predictions, for the optimal initial temperature and the time-performance trade-off, are also empirically validated on the partitioning problem. The significance of the temperature reduction factor, however, is much smaller than predicted. Key principles of the analysis are illustrated with a problem small enough to be analyzed numerically.