Strong self-concordance and sampling

Motivated by the Dikin walk, we develop aspects of the interior-point theory for sampling in high dimension. Specifically, we introduce the notions of strong self-concordance and symmetry for a barrier. These properties imply that the Dikin walk defined using a strongly self-concordant barrier with symmetry parameter ν mixes in Õ(nν) steps from a warm start for a convex body in ℝn. For many natural barriers, ν is roughly bounded by ν, the standard self-concordance parameter. We also show that these properties hold for the Lee-Sidford barrier. As a consequence, we obtain the first walk that mixes in Õ(n2) steps for an arbitrary polytope in ℝn. Strong self-concordance for other barriers leads to an interesting (and unexpected) connection — for the universal and entropic barriers, it is implied by the KLS conjecture.

[1]  Martin J. Wainwright,et al.  Fast MCMC Sampling Algorithms on Polytopes , 2017, J. Mach. Learn. Res..

[2]  Santosh S. Vempala,et al.  Hit-and-run from a corner , 2004, STOC '04.

[3]  Roland Hildebrand,et al.  Canonical Barriers on Convex Cones , 2014, Math. Oper. Res..

[4]  Santosh S. Vempala,et al.  Solving convex programs by random walks , 2004, JACM.

[5]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[6]  Santosh S. Vempala,et al.  Fast Algorithms for Logconcave Functions: Sampling, Rounding, Integration and Optimization , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[7]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[8]  M. Simonovits,et al.  Random walks and an O * ( n 5 ) volume algorithm for convex bodies , 1997 .

[9]  Santosh S. Vempala,et al.  Simulated annealing in convex bodies and an O*(n4) volume algorithm , 2006, J. Comput. Syst. Sci..

[10]  S. Vempala Geometric Random Walks: a Survey , 2007 .

[11]  Jonah Sherman,et al.  Nearly Maximum Flows in Nearly Linear Time , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[12]  Santosh S. Vempala,et al.  The geometry of logconcave functions and sampling algorithms , 2007, Random Struct. Algorithms.

[13]  László Lovász,et al.  Hit-and-run mixes fast , 1999, Math. Program..

[14]  Hariharan Narayanan,et al.  Random walks on polytopes and an affine interior point method for linear programming , 2009, STOC '09.

[15]  Aleksander Madry,et al.  Fast Approximation Algorithms for Cut-Based Problems in Undirected Graphs , 2010, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science.

[16]  Jonah Sherman,et al.  Area-convexity, l∞ regularization, and undirected multicommodity flow , 2017, STOC.

[17]  Aleksander Madry,et al.  Navigating Central Path with Electrical Flows: From Flows to Matchings, and Back , 2013, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[18]  Sébastien Bubeck,et al.  The entropic barrier: a simple and optimal universal self-concordant barrier , 2014, COLT.

[19]  Santosh S. Vempala,et al.  Eldan's Stochastic Localization and the KLS Hyperplane Conjecture: An Improved Lower Bound for Expansion , 2016, 2017 IEEE 58th Annual Symposium on Foundations of Computer Science (FOCS).

[20]  Nisheeth K. Vishnoi,et al.  The mixing time of the Dikin walk in a polytope - A simple proof , 2016, Oper. Res. Lett..

[21]  Miklós Simonovits,et al.  Random Walks in a Convex Body and an Improved Volume Algorithm , 1993, Random Struct. Algorithms.

[22]  Pravin M. Vaidya,et al.  Speeding-up linear programming using fast matrix multiplication , 1989, 30th Annual Symposium on Foundations of Computer Science.

[23]  Shang-Hua Teng,et al.  Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs , 2010, STOC '11.

[24]  Ronen Eldan,et al.  Thin Shell Implies Spectral Gap Up to Polylog via a Stochastic Localization Scheme , 2012, 1203.0893.

[25]  Yin Tat Lee,et al.  Efficient Inverse Maintenance and Faster Algorithms for Linear Programming , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[26]  Miklós Simonovits,et al.  Random walks and an O*(n5) volume algorithm for convex bodies , 1997, Random Struct. Algorithms.

[27]  YuBin,et al.  Fast MCMC sampling algorithms on polytopes , 2018 .

[28]  Hariharan Narayanan,et al.  John’s walk , 2018, Advances in Applied Probability.

[29]  Richard Peng,et al.  Approximate Undirected Maximum Flows in O(mpolylog(n)) Time , 2014, SODA.

[30]  Santosh S. Vempala,et al.  Geodesic walks in polytopes , 2016, STOC.

[31]  Yin Tat Lee,et al.  An Almost-Linear-Time Algorithm for Approximate Max Flow in Undirected Graphs, and its Multicommodity Generalizations , 2013, SODA.

[32]  Santosh S. Vempala,et al.  Convergence rate of Riemannian Hamiltonian Monte Carlo and faster polytope volume computation , 2017, STOC.

[33]  Pravin M. Vaidya,et al.  A new algorithm for minimizing convex functions over convex sets , 1996, Math. Program..

[34]  Nisheeth K. Vishnoi,et al.  Faster Polytope Rounding, Sampling, and Volume Computation via a Sub-Linear Ball Walk , 2019, 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS).

[35]  Yin Tat Lee,et al.  Path Finding Methods for Linear Programming: Solving Linear Programs in Õ(vrank) Iterations and Faster Algorithms for Maximum Flow , 2014, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[36]  James Renegar,et al.  A polynomial-time algorithm, based on Newton's method, for linear programming , 1988, Math. Program..

[37]  Yin Tat Lee,et al.  Solving Linear Programs with Sqrt(rank) Linear System Solves , 2019, ArXiv.