Efficient hybrid methods for global continuous optimization based on simulated annealing

We introduce several hybrid methods for global continuous optimization. They combine simulated annealing and a local proximal bundle method. Traditionally, the simplest hybrid of a global and a local solver is to call the local solver after the global one, but this does not necessarily produce good results. Besides, using efficient gradient-based local solvers implies that the hybrid can only be applied to differentiable problems. We show several ways how to integrate the local solver as a genuine part of simulated annealing to enable both efficient and reliable solution processes. When using the proximal bundle method as a local solver, it is possible to solve even nondifferentiable problems. The numerical tests show that the hybridization can improve both the efficiency and the reliability of simulated annealing.

[1]  Lester Ingber,et al.  Adaptive simulated annealing (ASA): Lessons learned , 2000, ArXiv.

[2]  Krzysztof C. Kiwiel,et al.  Proximity control in bundle methods for convex nondifferentiable minimization , 1990, Math. Program..

[3]  P. Neittaanmäki,et al.  Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control , 1992 .

[4]  L. Ingber.  Adaptive Simulated Annealing (ASA) , 1993 .

[5]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[6]  M. Montaz Ali,et al.  A direct search variant of the simulated annealing algorithm for optimization involving continuous variables , 2002, Comput. Oper. Res..

[7]  P. Preux,et al.  Towards hybrid evolutionary algorithms , 1999 .

[8]  C. Lemaréchal.  Nondifferentiable optimization , 1989 .

[9]  M. Locatelli.  Simulated Annealing Algorithms for Continuous Global Optimization: Convergence Conditions , 2000 .

[10]  Linet Özdamar,et al.  Experiments with new stochastic global optimization search techniques , 2000, Comput. Oper. Res..

[11]  Lester Ingber,et al.  Simulated annealing: Practice versus theory , 1993 .

[12]  El-Ghazali Talbi,et al.  A Taxonomy of Hybrid Metaheuristics , 2002, J. Heuristics.

[13]  P. Siarry,et al.  FITTING OF TABU SEARCH TO OPTIMIZE FUNCTIONS OF CONTINUOUS VARIABLES , 1997 .

[14]  Steven G. Louie,et al.  A Monte carlo simulated annealing approach to optimization over continuous variables , 1984 .

[15]  P. Pardalos,et al.  Handbook of global optimization , 1995 .

[16]  Theodore B. Trafalis,et al.  A novel metaheuristics approach for continuous global optimization , 2002, J. Glob. Optim..

[17]  Patrick Siarry,et al.  Tabu Search applied to global optimization , 2000, Eur. J. Oper. Res..

[18]  Der-San Chen,et al.  Continuous optimization by a variant of simulated annealing , 1996, Comput. Optim. Appl..

[19]  K. Miettinen,et al.  Quasi-random initial population for genetic algorithms , 2004 .

[20]  K. Kiwiel.  A Method for Solving Certain Quadratic Programming Problems Arising in Nonsmooth Optimization , 1986 .

[21]  John Yen,et al.  A hybrid approach to modeling metabolic systems using a genetic algorithm and simplex method , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[22]  Roberto Battiti,et al.  The continuous reactive tabu search: Blending combinatorial optimization and stochastic search for global optimization , 1996, Ann. Oper. Res..

[23]  Saïd Salhi,et al.  A hybrid algorithm for identifying global and local minima when optimizing functions with many minima , 2004, Eur. J. Oper. Res..

[24]  M. Locatelli.  Simulated Annealing Algorithms for Continuous Global Optimization , 2002 .

[25]  William L. Goffe,et al.  SIMANN: FORTRAN module to perform Global Optimization of Statistical Functions with Simulated Annealing , 1992 .

[26]  Patrick Siarry,et al.  A hybrid method combining continuous tabu search and Nelder-Mead simplex algorithms for the global optimization of multiminima functions , 2005, Eur. J. Oper. Res..

[27]  Patrick Siarry,et al.  Enhanced simulated annealing for globally minimizing functions of many-continuous variables , 1997, TOMS.

[28]  Panos M. Pardalos,et al.  Recent Advances in Global Optimization , 1991 .

[29]  Masao Fukushima,et al.  Hybrid simulated annealing and direct search method for nonlinear unconstrained global optimization , 2002, Optim. Methods Softw..

[30]  C. Storey,et al.  Aspiration Based Simulated Annealing Algorithm , 1997, J. Glob. Optim..

[31]  Kaisa Miettinen,et al.  Globally convergent limited memory bundle method for large-scale nonsmooth optimization , 2007, Math. Program..

[32]  Wesley E. Snyder,et al.  Optimization of functions with many minima , 1991, IEEE Trans. Syst. Man Cybern..

[33]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[34]  Kaisa Miettinen,et al.  New limited memory bundle method for large-scale nonsmooth optimization , 2004, Optim. Methods Softw..

[35]  F. Aluffi-Pentini,et al.  Global optimization and stochastic differential equations , 1985 .

[36]  Emile H. L. Aarts,et al.  Global optimization and simulated annealing , 1991, Math. Program..

[37]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[38]  Huanwen Tang,et al.  Global optimizations and tabu search based on memory , 2004, Appl. Math. Comput..

[39]  P. Siarry,et al.  A genetic algorithm with real-value coding to optimize multimodal continuous functions , 2001 .

[40]  Marco Locatelli,et al.  Convergence of a Simulated Annealing Algorithm for Continuous Global Optimization , 2000, J. Glob. Optim..