Optimal friction matrix for underdamped Langevin sampling

A systematic procedure for optimising the friction coefficient in underdamped Langevin dynamics as a sampling tool is given by taking the gradient of the associated asymptotic variance with respect to friction. We give an expression for this gradient in terms of the solution to an appropriate Poisson equation and show that it can be approximated by short simulations of the associated first variation/tangent process under concavity assumptions on the log density. Our algorithm is applied to the estimation of posterior means in Bayesian inference problems and reduced variance is demonstrated when compared to the original underdamped and overdamped Langevin dynamics in both full and stochastic gradient cases.

[1]  Murat A. Erdogdu,et al.  On the Ergodicity, Bias and Asymptotic Normality of Randomized Midpoint Sampling Method , 2020, NeurIPS.

[2]  G. Pavliotis,et al.  On the Generalized Langevin Equation for Simulated Annealing , 2020, SIAM/ASA Journal on Uncertainty Quantification.

[3]  B. Leimkuhler,et al.  Rational Construction of Stochastic Numerical Methods for Molecular Sampling , 2012, 1203.5428.

[4]  Radford M. Neal.  Bayesian Learning via Stochastic Dynamics , 1992, NIPS.

[5]  Optimal Scaling of Random-Walk Metropolis Algorithms on General Target Distributions , 2019 .

[6]  Euhanna Ghadimi,et al.  Global convergence of the Heavy-ball method for convex optimization , 2014, 2015 European Control Conference (ECC).

[7]  A. Horowitz.  The second order Langevin equation and numerical simulations , 1987 .

[8]  Padhraic Smyth,et al.  Approximate Slice Sampling for Bayesian Posterior Inference , 2014, AISTATS.

[9]  B. Leimkuhler,et al.  The computation of averages from equilibrium and nonequilibrium Langevin molecular dynamics , 2013, 1308.5814.

[10]  A. Guillin,et al.  Trend to equilibrium and particle approximation for a weakly selfconsistent Vlasov-Fokker-Planck equation , 2009, 0906.1417.

[11]  James Foster,et al.  The shifted ODE method for underdamped Langevin MCMC , 2021, ArXiv.

[12]  Christopher Nemeth,et al.  Control variates for stochastic gradient MCMC , 2017, Statistics and Computing.

[13]  R. Bhattacharya.  On the functional central limit theorem and the law of the iterated logarithm for Markov processes , 1982 .

[14]  Alain Durmus,et al.  Uniform minorization condition and convergence bounds for discretizations of kinetic Langevin dynamics , 2021, ArXiv.

[15]  Martin J. Wainwright,et al.  Log-concave sampling: Metropolis-Hastings algorithms are fast! , 2018, COLT.

[16]  Alain Durmus,et al.  High-dimensional Bayesian inference via the unadjusted Langevin algorithm , 2016, Bernoulli.

[17]  Pierre Monmarch'e,et al.  High-dimensional MCMC with a standard splitting scheme for the underdamped Langevin diffusion. , 2020, Electronic Journal of Statistics.

[18]  J. Voss,et al.  Analysis of SPDEs arising in path sampling. Part I: The Gaussian case , 2005 .

[19]  G. A. Pavliotis,et al.  Asymptotic analysis of the Green–Kubo formula , 2010, 1002.4103.

[20]  Vincent Danos,et al.  Langevin Dynamics with Variable Coefficients and Nonconservative Forces: From Stationary States to Numerical Methods , 2017, Entropy.

[21]  S. Duane,et al.  Hybrid Monte Carlo , 1987 .

[22]  J. M. Sanz-Serna,et al.  Wasserstein distance estimates for the distributions of numerical approximations to ergodic stochastic differential equations , 2021, J. Mach. Learn. Res..

[23]  Zofia Trstanova,et al.  Langevin Dynamics With General Kinetic Energies , 2016, Multiscale Model. Simul..

[24]  Woo Kyun Kim,et al.  Effects of Langevin friction and time steps in the molecular dynamics simulation of nanoindentation , 2020 .

[25]  D. Belomestny,et al.  Variance reduction for Markov chains with application to MCMC , 2020, Stat. Comput..

[26]  A. Doucet,et al.  Piecewise-Deterministic Markov Chain Monte Carlo , 2017, 1707.05296.

[27]  Djalil CHAFAÏ,et al.  Central limit theorems for additive functionals of ergodic Markov diffusions processes , 2011, 1104.2198.

[28]  M. Parrinello,et al.  Accurate sampling using Langevin dynamics. , 2007, Physical review. E, Statistical, nonlinear, and soft matter physics.

[29]  G. Roberts,et al.  MCMC methods for diffusion bridges , 2008 .

[30]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[31]  A. Stuart,et al.  MCMC methods for sampling function space , 2009 .

[32]  Art B. Owen,et al.  Statistically Efficient Thinning of a Markov Chain Sampler , 2015, ArXiv.

[33]  B. Leimkuhler,et al.  Robust and efficient configurational molecular sampling via Langevin dynamics. , 2013, The Journal of chemical physics.

[34]  Mingyi Hong,et al.  On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization , 2018, ICLR.

[35]  Michael I. Jordan,et al.  Underdamped Langevin MCMC: A non-asymptotic analysis , 2017, COLT.

[36]  Vladas Sidoravicius,et al.  Stochastic Processes and Applications , 2007 .

[37]  Arnaud Doucet,et al.  On Markov chain Monte Carlo methods for tall data , 2015, J. Mach. Learn. Res..

[38]  Martin Hairer,et al.  Sampling conditioned hypoelliptic diffusions , 2009, 0908.0162.

[39]  M. Betancourt,et al.  The Geometric Foundations of Hamiltonian Monte Carlo , 2014, 1410.5110.

[40]  Stephen J. Wright,et al.  Random Coordinate Underdamped Langevin Monte Carlo , 2020, AISTATS.

[41]  Malliavin matrix of degenerate SDE and gradient estimate , 2013, 1308.5776.

[42]  Gabriel Stoltz,et al.  Partial differential equations and stochastic methods in molecular dynamics* , 2016, Acta Numerica.

[43]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[44]  G. Stoltz,et al.  Spectral methods for Langevin dynamics and associated error estimates , 2017, 1702.04718.

[45]  Martin J. Wainwright,et al.  High-Order Langevin Diffusion Yields an Accelerated MCMC Algorithm , 2019, J. Mach. Learn. Res..

[46]  M. Röckner,et al.  A Concise Course on Stochastic Partial Differential Equations , 2007 .

[47]  A. Doucet,et al.  The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method , 2015, 1510.02451.

[48]  Centro internazionale matematico estivo. Session,et al.  Stochastic PDE's and Kolmogorov equations in infinite dimensions : lectures given at the 2nd session of the Centro Internazionale Matematico Estivo (C.I.M.E.) held in Cetraro, Italy, August 24- September 1, 1998 , 1999 .

[49]  J. Teichmann.  Calculating the Greeks by cubature formulae , 2004, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[50]  Arnak S. Dalalyan,et al.  On sampling from a log-concave density using kinetic Langevin diffusions , 2018, Bernoulli.

[51]  L. Hörmander.  Hypoelliptic second order differential equations , 1967 .

[52]  G. Pavliotis,et al.  Using Perturbed Underdamped Langevin Dynamics to Efficiently Sample from Probability Distributions , 2017, Journal of Statistical Physics.

[53]  Christophe Andrieu,et al.  A tutorial on adaptive MCMC , 2008, Stat. Comput..

[54]  Pierre-Louis Lions,et al.  Applications of Malliavin calculus to Monte Carlo methods in finance , 1999, Finance Stochastics.

[55]  T. Lelièvre,et al.  An efficient sampling algorithm for variational Monte Carlo. , 2006, The Journal of chemical physics.

[56]  S. Redon,et al.  Error Analysis of Modified Langevin Dynamics , 2016, 1601.07411.

[57]  Boris Polyak.  Some methods of speeding up the convergence of iteration methods , 1964 .

[58]  A. Stuart,et al.  ANALYSIS OF SPDES ARISING IN PATH SAMPLING PART II: THE NONLINEAR CASE , 2006, math/0601092.

[59]  Leah F. South,et al.  Regularised Zero-Variance Control Variates for High-Dimensional Variance Reduction , 2018 .

[60]  E. Vanden-Eijnden,et al.  Pathwise accuracy and ergodicity of metropolized integrators for SDEs , 2009, 0905.4218.

[61]  Michael I. Jordan,et al.  On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo , 2018, ICML.

[62]  Yee Whye Teh,et al.  Bayesian Learning via Stochastic Gradient Langevin Dynamics , 2011, ICML.

[63]  Robert D. Skeel,et al.  Choice of damping coefficient in Langevin dynamics , 2021, The European Physical Journal B.

[64]  Stéphane Redon,et al.  Estimating the speed-up of adaptively restrained Langevin dynamics , 2016, J. Comput. Phys..

[65]  L. Mones,et al.  Some Remarks on Preconditioning Molecular Dynamics , 2016, 1612.05435.

[66]  B. Delyon,et al.  Simulation of conditioned diffusion and application to parameter estimation , 2006 .

[67]  D. Dittmar.  Slice Sampling , 2000 .

[68]  Tomasz Komorowski,et al.  Fluctuations in Markov Processes , 2012 .

[69]  Michael I. Jordan,et al.  Is there an analog of Nesterov acceleration for gradient-based MCMC? , 2021 .

[70]  P. Protter.  Stochastic integration and differential equations , 1990 .

[71]  J. Wehr,et al.  The Smoluchowski-Kramers Limit of Stochastic Differential Equations with Arbitrary State-Dependent Friction , 2014, 1404.2330.

[72]  P. Fearnhead,et al.  The Zig-Zag process and super-efficient sampling for Bayesian analysis of big data , 2016, The Annals of Statistics.

[73]  A. Guillin,et al.  Degenerate Fokker–Planck equations: Bismut formula, gradient estimate and Harnack inequality , 2011, 1103.2817.

[74]  Yin Tat Lee,et al.  The Randomized Midpoint Method for Log-Concave Sampling , 2019, NeurIPS.

[75]  Philip H. Demp,et al.  A mathematical perspective , 2001 .

[76]  G. Pavliotis,et al.  Variance Reduction Using Nonreversible Langevin Samplers , 2015, Journal of statistical physics.

[77]  Yee Whye Teh,et al.  Exploration of the (Non-)Asymptotic Bias and Variance of Stochastic Gradient Langevin Dynamics , 2016, J. Mach. Learn. Res..

[78]  Christopher Nemeth,et al.  Stochastic Gradient Markov Chain Monte Carlo , 2019, Journal of the American Statistical Association.

[79]  Tao Tang,et al.  Combined Hermite spectral-finite difference method for the Fokker-Planck equation , 2002, Math. Comput..

[80]  Pierre Monmarché.  Almost sure contraction for diffusions on R. Application to generalised Langevin diffusions , 2020 .

[81]  A. Horowitz.  A generalized guided Monte Carlo algorithm , 1991 .

[82]  B. Guo,et al.  Spectral Methods and Their Applications , 1998 .

[83]  Quanquan Gu,et al.  On the Convergence of Hamiltonian Monte Carlo with Stochastic Gradients , 2021, ICML.