Theoretically Grounded Acceleration Techniques for Simulated Annealing

Simulated annealing (SA) is a generic optimization method whose popularity stems from its simplicity and its global convergence properties; it emulates the physical process of annealing whereby a solid is heated and then cooled down to eventually reach a minimum energy configuration. Although successfully applied to many difficult problems, SA is widely reported to converge very slowly, and it is common practice to relax some of its convergence conditions as well as to allow extra freedom in its design. However, variations on the theme of annealing usually come without optimal convergence guarantees.

[1]  Pierre-Jean Reissman,et al.  On Simulated annealing with temperature-dependent energy and temperature-dependent communication , 2011 .

[2]  O. Catoni.  Simulated annealing algorithms and Markov chains with rare transitions , 1999 .

[3]  P. Moral,et al.  On the Convergence and the Applications of the Generalized Simulated Annealing , 1999 .

[4]  M. Émery,et al.  Seminaire de Probabilites XXXIII , 1999 .

[5]  Pierre-Jean Reissman,et al.  From simulated annealing to stochastic continuation: a new trend in combinatorial optimization , 2012, Journal of Global Optimization.

[6]  J Besag,et al.  DISCUSSION ON THE MEETING ON THE GIBBS SAMPLER AND OTHER MARKOV CHAIN-MONTE CARLO METHODS , 1993 .

[7]  Sheldon Howard Jacobson,et al.  The Theory and Practice of Simulated Annealing , 2003, Handbook of Metaheuristics.

[8]  B. Gidas.  Nonstationary Markov chains and convergence of the annealing algorithm , 1985 .

[9]  Isabelle E. Magnin,et al.  Stochastic nonlinear image restoration using the wavelet transform , 2003, IEEE Trans. Image Process..

[10]  M. Desai.  Some results characterizing the finite time behaviour of the simulated annealing algorithm , 1999 .

[11]  P. C. Schuur,et al.  Classification of Acceptance Criteria for the Simulated Annealing Algorithm , 1997, Math. Oper. Res..

[12]  Bruce E. Hajek,et al.  Cooling Schedules for Optimal Annealing , 1988, Math. Oper. Res..

[13]  Gabriele Grillo,et al.  Simulated annealing with time-dependent energy function , 1993 .

[14]  Bruno Sixou,et al.  Optimal inverse treatment planning by stochastic continuation , 2011, 2011 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[15]  Isabelle E. Magnin,et al.  Optimization by Stochastic Continuation , 2010, SIAM J. Imaging Sci..

[16]  Mila Nikolova,et al.  Markovian reconstruction using a GNC approach , 1999, IEEE Trans. Image Process..

[17]  H. Cohn,et al.  Simulated Annealing: Searching for an Optimal Temperature Schedule , 1999, SIAM J. Optim..

[18]  T. Chiang,et al.  On the convergence rate of annealing processes , 1987 .

[19]  Olivier Catoni,et al.  Metropolis, Simulated Annealing, and Iterated Energy Transformation Algorithms: Theory and Experiments , 1996, J. Complex..

[20]  Sheldon Howard Jacobson,et al.  Analysis of Static Simulated Annealing Algorithms , 2002 .

[21]  Isabelle E. Magnin,et al.  A Stochastic Continuation Approach to Piecewise Constant Reconstruction , 2007, IEEE Transactions on Image Processing.

[22]  Chiang Tzuu-Shuh,et al.  On the convergence rate of annealing processes , 1988 .

[23]  O. Catoni.  Rough Large Deviation Estimates for Simulated Annealing: Application to Exponential Schedules , 1992 .

[24]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[25]  Mark Fielding,et al.  Simulated Annealing With An Optimal Fixed Temperature , 2000, SIAM J. Optim..

[26]  Isabelle E. Magnin,et al.  3-D reconstruction from a few radiographs using the metropolis dynamics with annealing , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[27]  Robert Azencott,et al.  Simulated annealing : parallelization techniques , 1992 .

[28]  Mads Nielsen.  Graduated Nonconvexity by Functional Focusing , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Matthias Löwe.  Simulated annealing with time-dependent energy function via Sobolev inequalities , 1996 .

[30]  Isabelle E. Magnin,et al.  Simulated annealing, acceleration techniques, and image restoration , 1999, IEEE Trans. Image Process..

[31]  Sheldon H. Jacobson,et al.  Finite-Time Performance Analysis of Static Simulated Annealing Algorithms , 2002, Comput. Optim. Appl..

[32]  J. Doob.  Stochastic processes , 1953 .

[33]  Sheldon H. Jacobson,et al.  On the convergence of generalized hill climbing algorithms , 2002, Discret. Appl. Math..