Simulated Annealing for Convex Optimization

We apply the method known as simulated annealing to the following problem in convex optimization: Minimize a linear function over an arbitrary convex set, where the convex set is specified only by a membership oracle. Using distributions from the Boltzmann-Gibbs family leads to an algorithm that needs only O*(√n) phases for instances in Rn. This gives an optimization algorithm that makes O*(n4.5) calls to the membership oracle, in the worst case, compared to the previous best guarantee of O*(n5). The benefits of using annealing here are surprising because such problems have no local minima that are not also global minima. Hence, we conclude that one of the advantages of simulated annealing, in addition to avoiding poor local minima, is that in these problems it converges faster to the minima that it finds. We also give a proof that under certain general conditions, the Boltzmann-Gibbs distributions are optimal for annealing on these convex problems.

[1]  David Applegate,et al.  Sampling and integration of near log-concave functions , 1991, STOC '91.

[2]  Nicholas G. Polson,et al.  Sampling from log-concave distributions , 1994 .

[3]  M. Simonovits,et al.  Random walks and an O * ( n 5 ) volume algorithm for convex bodies , 1997 .

[4]  Zelda B. Zabinsky,et al.  Stochastic Adaptive Search for Global Optimization , 2003 .

[5]  Mark Jerrum,et al.  Simulated annealing for graph bisection , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[6]  Gregory B. Sorkin,et al.  Efficient simulated annealing on fractal energy landscapes , 1991, Algorithmica.

[7]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[8]  Santosh S. Vempala,et al.  Hit-and-run from a corner , 2004, STOC '04.

[9]  Graham Wood,et al.  Stochastic Adaptive Search for Global Optimization , 2005 .

[10]  Bruce E. Hajek,et al.  Cooling Schedules for Optimal Annealing , 1988, Math. Oper. Res..

[11]  Miklós Simonovits,et al.  Random walks and an O*(n5) volume algorithm for convex bodies , 1997, Random Struct. Algorithms.

[12]  Santosh S. Vempala,et al.  Simulated annealing in convex bodies and an O*(n4) volume algorithm , 2006, J. Comput. Syst. Sci..

[13]  L. Lovász,et al.  Geometric Algorithms and Combinatorial Optimization , 1981 .

[14]  J. G. Pierce,et al.  Geometric Algorithms and Combinatorial Optimization , 2016 .

[15]  M. Rudelson.  Random Vectors in the Isotropic Position , 1996, math/9608208.

[16]  S. Vempala.  Geometric Random Walks: a Survey , 2007 .

[17]  Robert L. Smith,et al.  Improving Hit-and-Run for global optimization , 1993, J. Glob. Optim..

[18]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[19]  A. Prékopa.  On logarithmic concave measures and functions , 1973 .

[20]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[21]  Eric Vigoda,et al.  A polynomial-time approximation algorithm for the permanent of a matrix with non-negative entries , 2001, STOC '01.

[22]  S. Vempala,et al.  The geometry of logconcave functions and sampling algorithms , 2007 .

[23]  Eric Vigoda,et al.  A polynomial-time approximation algorithm for the permanent of a matrix with nonnegative entries , 2004, JACM.

[24]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[25]  Santosh S. Vempala,et al.  Simulated annealing in convex bodies and an O*(n/sup 4/) volume algorithm , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[26]  A. Prékopa.  Logarithmic concave measures with applications to stochastic programming , 1971 .

[27]  L. Leindler.  On a Certain Converse of Hölder’s Inequality , 1972 .

[28]  Santosh S. Vempala,et al.  Solving convex programs by random walks , 2004, JACM.

[29]  A. Dinghas.  Über eine Klasse superadditiver Mengenfunktionale von Brunn-Minkowski-Lusternikschem Typus , 1957 .

[30]  Santosh S. Vempala,et al.  The geometry of logconcave functions and sampling algorithms , 2007, Random Struct. Algorithms.