The query complexity of sampling from strongly log-concave distributions in one dimension

We establish the first tight lower bound of $\Omega(\log\log\kappa)$ on the query complexity of sampling from the class of strongly log-concave and log-smooth distributions with condition number $\kappa$ in one dimension. Whereas existing guarantees for MCMC-based algorithms scale polynomially in $\kappa$, we introduce a novel algorithm based on rejection sampling that closes this doubly exponential gap.

[1]  S. Schmidler,et al.  Minimax Mixing Time of the Metropolis-Adjusted Langevin Algorithm for Log-Concave Sampling , 2021, J. Mach. Learn. Res..

[2]  Kevin Tian,et al.  Lower Bounds on Metropolized Sampling Methods for Well-Conditioned Distributions , 2021, NeurIPS.

[3]  Michael I. Jordan,et al.  Is there an analog of Nesterov acceleration for gradient-based MCMC? , 2021 .

[4]  Sinho Chewi,et al.  Optimal dimension dependence of the Metropolis-Adjusted Langevin Algorithm , 2020, COLT.

[5]  Stephen J. Wright,et al.  Random Coordinate Underdamped Langevin Monte Carlo , 2020, AISTATS.

[6]  Stephen J. Wright,et al.  Random Coordinate Langevin Monte Carlo , 2020, COLT.

[7]  Tyler Maunu,et al.  Exponential ergodicity of mirror-Langevin diffusions , 2020, NeurIPS.

[8]  Yu Cao,et al.  Complexity of randomized algorithms for underdamped Langevin dynamics , 2020, Communications in Mathematical Sciences.

[9]  Anna Korba,et al.  The Wasserstein Proximal Gradient Algorithm , 2020, NeurIPS.

[10]  Philip M. Long,et al.  Oracle lower bounds for stochastic gradient sampling algorithms , 2020, Bernoulli.

[11]  Rong Ge,et al.  Estimating normalizing constants for log-concave distributions: algorithms and lower bounds , 2019, STOC.

[12]  Andre Wibisono,et al.  Proximal Langevin Algorithm: Rapid Convergence Under Isoperimetry , 2019, ArXiv.

[13]  Yin Tat Lee,et al.  The Randomized Midpoint Method for Log-Concave Sampling , 2019, NeurIPS.

[14]  Alain Durmus,et al.  Analysis of Langevin Monte Carlo via Convex Optimization , 2018, J. Mach. Learn. Res..

[15]  Espen Bernton,et al.  Langevin Monte Carlo and JKO splitting , 2018, COLT.

[16]  Andre Wibisono,et al.  Sampling as optimization in the space of measures: The Langevin dynamics as a composite optimization problem , 2018, COLT.

[17]  Arnak S. Dalalyan,et al.  User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient , 2017, Stochastic Processes and their Applications.

[18]  É. Moulines,et al.  Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm , 2015, 1507.05021.

[19]  A. Dalalyan.  Theoretical guarantees for approximate sampling from smooth and log‐concave densities , 2014, 1412.7392.

[20]  Sébastien Bubeck.  Convex Optimization: Algorithms and Complexity , 2014, Found. Trends Mach. Learn..

[21]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[22]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[23]  László Lovász,et al.  Hit-and-run mixes fast , 1999, Math. Program..

[24]  Robert L. Smith,et al.  Hit-and-Run Algorithms for Generating Multivariate Distributions , 1993, Math. Oper. Res..

[25]  R. D. Gordon.  Values of Mills' Ratio of Area to Bounding Ordinate and of the Normal Probability Integral for Large Values of the Argument , 1941 .

[26]  Télécom Paristech.  NONASYMPTOTIC CONVERGENCE ANALYSIS FOR THE UNADJUSTED LANGEVIN ALGORITHM1 , 2017 .

[27]  Arkadi Nemirovski,et al.  EFFICIENT METHODS IN CONVEX PROGRAMMING , 2007 .

[28]  S. Vempala,et al.  Hit-and-Run is Fast and Fun 1 , 2003 .

[29]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[30]  Grace L. Yang,et al.  Asymptotics In Statistics , 1990 .

[31]  L. L. Cam,et al.  Asymptotic Methods In Statistical Decision Theory , 1986 .

[32]  Jerzy Seidler,et al.  Problem Complexity and Method Efficiency in Optimization , 1984 .

[33]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .