Simulated Annealing: Searching for an Optimal Temperature Schedule

A sizable part of the theoretical literature on simulated annealing deals with a property called convergence, which asserts that the simulated annealing chain is in the set of global minimum states of the objective function with probability tending to 1. However, in practice, the convergent algorithms are considered too slow, whereas a number of nonconvergent ones are usually preferred. We attempt a detailed analysis of various temperature schedules. Examples will be given of when it is both practically and theoretically justified to use boiling, fixed temperature, or even fast cooling schedules which have a small probability of reaching global minima. Applications to traveling salesman problems of various sizes are also given.

[1]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[2]  O. Barndorff-Nielsen,et al.  On the Limit Behaviour of Extreme Order Statistics , 1963 .

[3]  G. L. Thompson,et al.  A Heuristic Approach to Solving Travelling Salesman Problems , 1964 .

[4]  Patrick D. Krolak,et al.  A man-machine approach toward solving the traveling salesman problem , 1970, DAC '70.

[5]  Brian W. Kernighan,et al.  An Effective Heuristic Algorithm for the Traveling-Salesman Problem , 1973, Oper. Res..

[6]  Martin Grötschel,et al.  Polyedrische Charakterisierungen kombinatorischer Optimierungsprobleme , 1977 .

[7]  H. Cohn.  On a paper by Doeblin on non-homogeneous Markov chains , 1981, Advances in Applied Probability.

[8]  On a class of non-homogeneous Markov chains , 1982 .

[9]  Valerie Isham,et al.  Non‐Negative Matrices and Markov Chains , 1983 .

[10]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[11]  Scott Kirkpatrick,et al.  Optimization by simulated annealing: Quantitative studies , 1984 .

[12]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Ehl Emile Aarts,et al.  Statistical cooling : a general approach to combinatorial optimization problems , 1985 .

[14]  Sanjoy Mitter,et al.  Analysis of simulated annealing for optimization , 1985, 1985 24th IEEE Conference on Decision and Control.

[15]  Alistair I. Mees,et al.  Convergence of an annealing algorithm , 1986, Math. Program..

[16]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[17]  T. Chiang,et al.  On the convergence rate of annealing processes , 1987 .

[18]  G. B. Smith,et al.  Preface to S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images” , 1987 .

[19]  Daniel Patrick Connors.  Balance of Recurrence Order in Time-Inhomogeneous Markov Chains With Applications to Simulated Annealing , 1988 .

[20]  Chiang Tzuu-Shuh,et al.  On the convergence rate of annealing processes , 1988 .

[21]  Bruce E. Hajek,et al.  Cooling Schedules for Optimal Annealing , 1988, Math. Oper. Res..

[22]  H. Cohn.  PRODUCTS OF STOCHASTIC MATRICES AND APPLICATIONS , 1989 .

[23]  David Connolly.  An improved annealing scheme for the QAP , 1990 .

[24]  Martin Grötschel,et al.  Solution of large-scale symmetric travelling salesman problems , 1991, Math. Program..

[25]  C. Hwang,et al.  Singular perturbed Markov chains and exact behaviors of simulated annealing processes , 1992 .

[26]  P. Pokarowski,et al.  Tail events of some nonhomogeneous Markov chains , 1995 .

[27]  Russell Greiner,et al.  PALO: A Probabilistic Hill-Climbing Algorithm , 1996, Artif. Intell..