Variance Reduction Using Nonreversible Langevin Samplers

A standard approach to computing expectations with respect to a given target measure is to introduce an overdamped Langevin equation which is reversible with respect to the target distribution, and to approximate the expectation by a time-averaging estimator. As has been noted in recent papers [30, 37, 61, 72], introducing an appropriately chosen nonreversible component to the dynamics is beneficial, both in terms of reducing the asymptotic variance and of speeding up convergence to the target distribution. In this paper we present a detailed study of the dependence of the asymptotic variance on the deviation from reversibility. Our theoretical findings are supported by numerical simulations.

[1]  L. Schmetterer.  Zeitschrift fur Wahrscheinlichkeitstheorie und Verwandte Gebiete. , 1963 .

[2]  G. Strang.  On the Construction and Comparison of Difference Schemes , 1968 .

[3]  P. Peskun,et al.  Optimum Monte-Carlo sampling using Markov chains , 1973 .

[4]  Inge S. Helland,et al.  Central Limit Theorems for Martingales with Discrete or Continuous Time , 1982 .

[5]  R. Bhattacharya.  On the functional central limit theorem and the law of the iterated logarithm for Markov processes , 1982 .

[6]  George Papanicolaou,et al.  Bounds for effective parameters of heterogeneous media by analytic continuation , 1983 .

[7]  Rabi Bhattacharya,et al.  A Central Limit Theorem for Diffusions with Periodic Coefficients , 1985 .

[8]  S. Varadhan,et al.  Central limit theorem for additive functionals of reversible Markov processes and applications to simple exclusions , 1986 .

[9]  D. Newton,et al.  ERGODIC THEOREMS (de Gruyter Studies in Mathematics 6) , 1986 .

[10]  A. Shiryaev,et al.  Limit Theorems for Stochastic Processes , 1987 .

[11]  Avellaneda,et al.  Stieltjes integral representation and effective diffusivity bounds for turbulent transport. , 1989, Physical review letters.

[12]  Homer F. Walker,et al.  Asymptotics of solute dispersion in periodic porous media , 1989 .

[13]  D. Talay,et al.  Expansion of the global error for numerical schemes solving stochastic differential equations , 1990 .

[14]  M. Yor,et al.  Continuous martingales and Brownian motion , 1990 .

[15]  Andrew J. Majda,et al.  An integral representation and bounds on the effective diffusivity in passive advection by laminar and turbulent flows , 1991 .

[16]  Andrew J. Majda,et al.  The Effect of Mean Flows on Enhanced Diffusivity in Transport by Incompressible Periodic Velocity Fields , 1993 .

[17]  C. Hwang,et al.  Accelerating Gaussian Diffusions , 1993 .

[18]  L. R.,et al.  A Survey of Foster-Lyapunov Techniquesfor General State Space Markov ProcessesS , 1993 .

[19]  S. Meyn,et al.  Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time processes , 1993, Advances in Applied Probability.

[20]  S. Meyn,et al.  Exponential and Uniform Ergodicity of Markov Processes , 1995 .

[21]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[22]  R. Tweedie,et al.  Exponential convergence of Langevin distributions and their discrete approximations , 1996 .

[23]  Radford M. Neal.  Sampling from multimodal distributions using tempered transitions , 1996, Stat. Comput..

[24]  R. Bhatia.  Matrix Analysis , 1996 .

[25]  Sean P. Meyn,et al.  A Liapounov bound for solutions of the Poisson equation , 1996 .

[26]  R. Bhattacharya.  Multiscale diffusion processes with periodic coefficients and an application to solute transport in porous media , 1999 .

[27]  A. Majda,et al.  SIMPLIFIED MODELS FOR TURBULENT DIFFUSION : THEORY, NUMERICAL MODELLING, AND PHYSICAL PHENOMENA , 1999 .

[28]  Heikki Haario,et al.  Adaptive proposal distribution for random walk Metropolis algorithm , 1999, Comput. Stat..

[29]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[30]  Radford M. Neal,et al.  ANALYSIS OF A NONREVERSIBLE MARKOV CHAIN SAMPLER , 2000 .

[31]  Antonietta Mira,et al.  Ordering and Improving the Performance of Monte Carlo Markov Chains , 2001 .

[32]  A. Veretennikov,et al.  On the poisson equation and diffusion approximation 3 , 2001, math/0506596.

[33]  G. Roberts,et al.  Langevin Diffusions and Metropolis-Hastings Algorithms , 2002 .

[34]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[35]  Jonathan C. Mattingly,et al.  Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise , 2002 .

[36]  Radford M. Neal.  Improving Asymptotic Variance of MCMC Estimators: Non-reversible Chains are Better , 2004, math/0407281.

[37]  A. Bovier,et al.  Metastability in Reversible Diffusion Processes I: Sharp Asymptotics for Capacities and Exit Times , 2004 .

[38]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[39]  P. Constantin,et al.  Diffusion and mixing in fluid flow , 2005 .

[40]  C. Hwang,et al.  Accelerating diffusions , 2005, math/0505245.

[41]  A. Bovier,et al.  Metastability in reversible diffusion processes II. Precise asymptotics for small eigenvalues , 2005 .

[42]  R. Douc,et al.  Subgeometric rates of convergence of f-ergodic strong Markov processes , 2006, math/0605791.

[43]  L. Lorenzi,et al.  Analytical Methods for Markov Semigroups , 2006 .

[44]  E. Hairer,et al.  Simulating Hamiltonian dynamics , 2006, Math. Comput..

[45]  Peter W. Glynn,et al.  Stochastic Simulation: Algorithms and Analysis , 2007 .

[46]  Michael Chertkov,et al.  Irreversible Monte Carlo Algorithms for Efficient Sampling , 2008, ArXiv.

[47]  Chii-Ruey Hwang,et al.  The behavior of the spectral gap under growing drift , 2009 .

[48]  G. A. Pavliotis,et al.  Asymptotic analysis of the Green–Kubo formula , 2010, 1002.4103.

[49]  Yi Sun,et al.  Improving the Asymptotic Performance of Markov Chain Monte-Carlo by Inserting Vortices , 2010, NIPS.

[50]  Andrew M. Stuart,et al.  Convergence of Numerical Time-Averaging and Stationary Measures via Poisson Equations , 2009, SIAM J. Numer. Anal..

[51]  T. Lelièvre,et al.  Free Energy Computations: A Mathematical Perspective , 2010 .

[52]  Radford M. Neal.  MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[53]  Djalil CHAFAÏ,et al.  Central limit theorems for additive functionals of ergodic Markov diffusions processes , 2011, 1104.2198.

[54]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[55]  Tomasz Komorowski,et al.  Fluctuations in Markov Processes: Time Symmetry and Martingale Approximation , 2012 .

[56]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[57]  James Martin,et al.  A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion , 2012, SIAM J. Sci. Comput..

[58]  General Construction of Irreversible Kernel in Markov Chain Monte Carlo , 2012, 1207.0258.

[59]  P. Dellaportas,et al.  Control variates for estimation based on reversible Markov chain Monte Carlo samplers , 2012 .

[60]  Simple implementation of Langevin dynamics neglecting detailed balance condition , 2013 .

[61]  K. Hukushima,et al.  An irreversible Markov-chain Monte Carlo method with skew detailed balance conditions , 2013 .

[62]  Bernard Helffer,et al.  Spectral Theory and its Applications , 2013 .

[63]  Christof Schütte,et al.  Metastability and Markov State Models in Molecular Dynamics Modeling, Analysis , 2016 .

[64]  G. Pavliotis,et al.  Optimal Non-reversible Linear Drift for the Convergence to Equilibrium of a Diffusion , 2012, 1212.0876.

[65]  Tony Lelievre,et al.  Two mathematical tools to analyze metastable stochastic processes , 2012, 1201.3775.

[66]  Sheng-Jhih Wu,et al.  Variance reduction for diffusions , 2014, 1406.4657.

[67]  T. Eisner,et al.  Ergodic Theorems , 2019, Probability.

[68]  K. Spiliopoulos,et al.  Variance reduction for irreversible Langevin samplers and diffusion on graphs , 2014, 1410.0255.

[69]  C. Hwang,et al.  Attaining the Optimal Gaussian Diffusion Acceleration , 2014 .

[70]  G. Pavliotis.  Stochastic Processes and Applications: Diffusion Processes, the Fokker-Planck and Langevin Equations , 2014 .

[71]  Guillaume Hennequin,et al.  Fast Sampling-Based Inference in Balanced Neuronal Networks , 2014, NIPS.

[72]  Masayuki Ohzeki,et al.  Langevin dynamics neglecting detailed balance condition. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[73]  Tianqi Chen,et al.  A Complete Recipe for Stochastic Gradient MCMC , 2015, NIPS.

[74]  K. Spiliopoulos,et al.  Irreversible Langevin samplers and variance reduction: a large deviations approach , 2014, 1404.0105.

[75]  Joris Bierkens,et al.  Non-reversible Metropolis-Hastings , 2014, Stat. Comput..