Complexity Analysis of a Sampling-Based Interior Point Method for Convex Optimization

We develop a short-step interior point method to optimize a linear function over a convex body assuming that one only knows a membership oracle for this body. The approach is based a sketch of a universal interior point method using the so-called entropic barrier. It is well known that the gradient and Hessian of the entropic barrier can be approximated by sampling from Boltzmann-Gibbs distributions and the entropic barrier was shown to be self-concordant. The analysis of our algorithm uses properties of the entropic barrier, mixing times for hit-and-run random walks, approximation quality guarantees for the mean and covariance of a log-concave distribution, and results on inexact Newton-type methods.

[1]  Sercan Yildiz,et al.  On "A Homogeneous Interior-Point Algorithm for Non-Symmetric Convex Conic Optimization" , 2017 .

[2]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[3]  Miklós Simonovits,et al.  Isoperimetric problems for convex bodies and a localization lemma , 1995, Discret. Comput. Geom..

[4]  B. Klartag.  On convex perturbations with a bounded isotropic constant , 2006 .

[5]  Santosh S. Vempala,et al.  Simulated Annealing for Convex Optimization , 2004 .

[6]  Robert L. Smith,et al.  Efficient Monte Carlo Procedures for Generating Points Uniformly Distributed over Bounded Regions , 1984, Oper. Res..

[7]  Miklós Simonovits,et al.  Random walks and an O*(n5) volume algorithm for convex bodies , 1997, Random Struct. Algorithms.

[8]  Etienne de Klerk,et al.  Worst-case convergence analysis of gradient and Newton methods through semidefinite programming performance estimation , 2017 .

[9]  S. Vempala,et al.  Hit-and-Run from a Corner , 2006 .

[10]  Robert L. Smith,et al.  Hit-and-Run Algorithms for Generating Multivariate Distributions , 1993, Math. Oper. Res..

[11]  Sébastien Bubeck,et al.  The entropic barrier: a simple and optimal universal self-concordant barrier , 2014, COLT.

[12]  Y. Peres,et al.  Markov Chains and Mixing Times: Second Edition , 2017 .

[13]  M. H. Wright.  The interior-point revolution in optimization: History, recent developments, and lasting consequences , 2004 .

[14]  Santosh S. Vempala,et al.  Simulated annealing in convex bodies and an O*(n4) volume algorithm , 2006, J. Comput. Syst. Sci..

[15]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[16]  Santosh S. Vempala,et al.  Fast Algorithms for Logconcave Functions: Sampling, Rounding, Integration and Optimization , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[17]  Santosh S. Vempala,et al.  The geometry of logconcave functions and sampling algorithms , 2007, Random Struct. Algorithms.

[18]  Yinyu Ye,et al.  A homogeneous interior-point algorithm for nonsymmetric convex conic optimization , 2014, Mathematical Programming.

[19]  James Renegar,et al.  A mathematical view of interior-point methods in convex optimization , 2001, MPS-SIAM series on optimization.

[20]  Osman Güler,et al.  Barrier Functions in Interior Point Methods , 1996, Math. Oper. Res..

[21]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[22]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[23]  M. Simonovits,et al.  Random walks and an O * ( n 5 ) volume algorithm for convex bodies , 1997 .

[24]  Elad Hazan,et al.  Faster Convex Optimization: Simulated Annealing with an Efficient Universal Barrier , 2015, ICML.

[25]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[26]  Robert L. Smith,et al.  Direction Choice for Accelerated Convergence in Hit-and-Run Sampling , 1998, Oper. Res..