Markov Chains with Rare Transitions and Simulated Annealing

We consider “approximate stationary” Markov chains in which the entries of the one-step transition probability matrix are known to be of different orders of magnitude and whose structure that is, the orders of magnitude of the transition probabilities does not change with time. For such Markov chains we present a method for generating order of magnitude estimates for the t-step transition probabilities, for any t. We then notice that algorithms of the simulated annealing type may be represented by Markov chains which are approximately stationary over fairly long time intervals. Using our results we obtain a characterization of the convergent “cooling” schedules for the most general class of algorithms of the simulated annealing type.

[1]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[2]  Marcel Coderch i Collell.  Multiple time scale approach to heirarchical aggregation of linear systems and finite state Markov processes , 1982 .

[3]  S. Sastry,et al.  Hierarchical aggregation of singularly perturbed finite state Markov processes , 1983 .

[4]  F. Delebecque.  A Reduction Process for Perturbed Markov Chains , 1983 .

[5]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[6]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  D. Mitra,et al.  Convergence and finite-time behavior of simulated annealing , 1985, 1985 24th IEEE Conference on Decision and Control.

[8]  B. Gidas.  Nonstationary Markov chains and convergence of the annealing algorithm , 1985 .

[9]  Sanjoy Mitter,et al.  Analysis of simulated annealing for optimization , 1985, 1985 24th IEEE Conference on Decision and Control.

[10]  S. Geman,et al.  Diffusions for global optimizations , 1986 .

[11]  C. Hwang,et al.  Diffusion for global optimization in R n , 1987 .

[12]  Bart W. Stuck,et al.  A Computer and Communication Network Performance Analysis Primer (Prentice Hall, Englewood Cliffs, NJ, 1985; revised, 1987) , 1987, Int. CMG Conference.

[13]  Alan S. Willsky,et al.  The reduction of perturbed Markov generators: an algorithm exposing the role of transient states , 1988, JACM.

[14]  Bruce E. Hajek,et al.  Cooling Schedules for Optimal Annealing , 1988, Math. Oper. Res..

[15]  Wendell H. Fleming,et al.  Stochastic differential systems, stochastic control theory and applications , 1988 .

[16]  J. Tsitsiklis.  A survey of large time asymptotics of simulated annealing algorithms , 1988 .