Role of current fluctuations in nonreversible samplers.

It is known that the distribution of nonreversible Markov processes breaking the detailed balance condition converges faster to the stationary distribution compared to reversible processes having the same stationary distribution. This is used in practice to accelerate Markov chain Monte Carlo algorithms that sample the Gibbs distribution by adding nonreversible transitions or nongradient drift terms. The breaking of detailed balance also accelerates the convergence of empirical estimators to their ergodic expectation in the long-time limit. Here, we give a physical interpretation of this second form of acceleration in terms of currents associated with the fluctuations of empirical estimators using the level 2.5 of large deviations, which characterizes the likelihood of density and current fluctuations in Markov processes. Focusing on diffusion processes, we show that there is accelerated convergence because estimator fluctuations arise in general with current fluctuations, leading to an added large deviation cost compared to the reversible case, which shows no current. We study the current fluctuation most likely to arise in conjunction with a given estimator fluctuation and provide bounds on the acceleration, based on approximations of this current. We illustrate these results for the Ornstein-Uhlenbeck process in two dimensions and the Brownian motion on the circle.

[1]  J. Elgin The Fokker-Planck Equation: Methods of Solution and Applications , 1984 .

[2]  R. Fonck,et al.  Flows ! , 2003 .

[3]  C. Monthus Large deviations for Markov processes with stochastic resetting: analysis via the empirical density and flows or via excursions between resets , 2020, Journal of Statistical Mechanics: Theory and Experiment.

[4]  R. Jack,et al.  Acceleration of Convergence to Equilibrium in Markov Chains by Breaking Detailed Balance , 2016, Journal of statistical physics.

[5]  S. Varadhan,et al.  Asymptotic evaluation of certain Markov process expectations for large time , 1975 .

[6]  J. P. Garrahan,et al.  Inferring dissipation from current fluctuations , 2017 .

[7]  Hugo Touchette,et al.  Variational and optimal control representations of conditioned and driven processes , 2015, 1506.05291.

[8]  K. Spiliopoulos,et al.  Variance reduction for irreversible Langevin samplers and diffusion on graphs , 2014, 1410.0255.

[9]  Gabriel Stoltz,et al.  Partial differential equations and stochastic methods in molecular dynamics* , 2016, Acta Numerica.

[10]  Udo Seifert,et al.  Universal bounds on current fluctuations. , 2015, Physical review. E.

[11]  A. Ichiki,et al.  Violation of detailed balance accelerates relaxation. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[12]  Robert L. Jack,et al.  Effective interactions and large deviations in stochastic processes , 2015, The European Physical Journal Special Topics.

[13]  Pedro L. Garrido,et al.  Thermodynamics of Currents in Nonequilibrium Diffusive Systems: Theory and Simulation , 2013, 1312.1246.

[14]  R. Jack,et al.  Canonical Structure and Orthogonality of Forces and Currents in Irreversible Markov Chains , 2017, Journal of Statistical Physics.

[15]  F. Bouchet,et al.  Perturbative Calculation of Quasi-Potential in Non-equilibrium Diffusions: A Mean-Field Example , 2015, 1509.03273.

[16]  A. Engel,et al.  Level 2 and level 2.5 large deviation functionals for systems with and without detailed balance , 2016, 1602.02545.

[17]  G. Pavliotis,et al.  Optimal Non-reversible Linear Drift for the Convergence to Equilibrium of a Diffusion , 2012, 1212.0876.

[18]  H. Touchette Introduction to dynamical large deviations of Markov processes , 2017, Physica A: Statistical Mechanics and its Applications.

[19]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[20]  Konstantinos Spiliopoulos,et al.  Improving the Convergence of Reversible Samplers , 2016 .

[21]  C. Monthus Revisiting classical and quantum disordered systems from the unifying perspective of large deviations , 2019, The European Physical Journal B.

[22]  Todd R. Gingrich,et al.  Dissipation Bounds All Steady-State Current Fluctuations. , 2015, Physical review letters.

[23]  Michael Chertkov,et al.  Irreversible Monte Carlo Algorithms for Efficient Sampling , 2008, ArXiv.

[24]  L. Bertini,et al.  Non Equilibrium Current Fluctuations in Stochastic Lattice Gases , 2006 .

[25]  H. Risken Solutions of the Fokker-Planck equation in detailed balance , 1972 .

[26]  G. Parmigiani Large Deviation Techniques in Decision, Simulation and Estimation , 1992 .

[27]  Michela Ottobre,et al.  Markov Chain Monte Carlo and Irreversibility , 2016 .

[28]  J. Gärtner On Large Deviations from the Invariant Measure , 1977 .

[29]  H. Touchette The large deviation approach to statistical mechanics , 2008, 0804.0327.

[30]  A. C. Barato,et al.  A Formal View on Level 2.5 Large Deviations and Fluctuation Relations , 2014, 1408.5033.

[31]  Peter Sollich,et al.  Large deviations and ensembles of trajectories in stochastic models , 2009, 0911.0211.

[32]  H. Risken Fokker-Planck Equation , 1996 .

[33]  C. Hwang,et al.  Accelerating diffusions , 2005, math/0505245.

[34]  G. Pavliotis,et al.  Using Perturbed Underdamped Langevin Dynamics to Efficiently Sample from Probability Distributions , 2017, Journal of Statistical Physics.

[35]  H. Touchette,et al.  Process interpretation of current entropic bounds , 2017, The European Physical Journal B.

[36]  U. Seifert Stochastic thermodynamics, fluctuation theorems and molecular machines , 2012, Reports on progress in physics. Physical Society.

[37]  C. Hwang,et al.  Accelerating Gaussian Diffusions , 1993 .

[38]  Joris Bierkens,et al.  Non-reversible Metropolis-Hastings , 2014, Stat. Comput..

[39]  C. Hwang,et al.  Attaining the Optimal Gaussian Diffusion Acceleration , 2014 .

[40]  Hugo Touchette,et al.  Nonequilibrium Markov Processes Conditioned on Large Deviations , 2014, 1405.5157.

[41]  G. Pavliotis,et al.  Variance Reduction Using Nonreversible Langevin Samplers , 2015, Journal of statistical physics.

[42]  Galin L. Jones On the Markov chain central limit theorem , 2004, math/0409112.

[43]  Martin Weigel,et al.  Non-reversible Monte Carlo simulations of spin models , 2011, Comput. Phys. Commun..

[44]  C. Landim,et al.  Macroscopic fluctuation theory , 2014, 1404.6466.

[45]  C. Monthus Large deviations for the density and current in non-equilibrium-steady-states on disordered rings , 2018, Journal of Statistical Mechanics: Theory and Experiment.

[46]  Jordan M. Horowitz,et al.  Quantifying dissipation using fluctuating currents , 2019, Nature Communications.

[47]  Jordan M. Horowitz,et al.  Inferring dissipation from current fluctuations , 2016 .

[48]  K. Spiliopoulos,et al.  Irreversible Langevin samplers and variance reduction: a large deviations approach , 2014, 1404.0105.

[49]  S. Dattagupta Stochastic Thermodynamics , 2021, Resonance.

[50]  Hugo Touchette,et al.  Nonequilibrium microcanonical and canonical ensembles and their equivalence. , 2013, Physical review letters.

[51]  H. Touchette,et al.  Large deviations of the current for driven periodic diffusions. , 2016, Physical review. E.

[52]  A. Faggionato,et al.  Flows, currents, and cycles for Markov Chains: large deviation asymptotics , 2014, 1408.5477.

[53]  Synge Todo,et al.  Markov chain Monte Carlo method without detailed balance. , 2010, Physical review letters.

[54]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[55]  J. Zimmer,et al.  Orthogonality of fluxes in general nonlinear reaction networks , 2019, Discrete & Continuous Dynamical Systems - S.

[56]  C. Hwang,et al.  Accelerating reversible Markov chains , 2013 .

[57]  Michael Chertkov,et al.  Stochastic Optimal Control as Non-equilibrium Statistical Mechanics: Calculus of Variations over Density and Current , 2013, ArXiv.

[58]  C. Maes,et al.  On and beyond entropy production: the case of Markov jump processes , 2007, 0709.4327.

[59]  Tomasz Komorowski,et al.  Fluctuations in Markov Processes , 2012 .

[60]  Sheng-Jhih Wu,et al.  Variance reduction for diffusions , 2014, 1406.4657.

[61]  Koji Hukushima,et al.  Eigenvalue analysis of an irreversible random walk with skew detailed balance conditions. , 2015, Physical review. E.

[62]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[63]  Djalil CHAFAÏ,et al.  Central limit theorems for additive functionals of ergodic Markov diffusions processes , 2011, 1104.2198.

[64]  Peter W. Glynn,et al.  Stochastic Simulation: Algorithms and Analysis , 2007 .

[65]  Upanshu Sharma,et al.  Variational structures beyond gradient flows: a macroscopic fluctuation-theory perspective , 2021, 2103.14384.

[66]  D. M. Renger Flux Large Deviations of Independent and Reacting Particle Systems, with Implications for Macroscopic Fluctuation Theory , 2018, Journal of Statistical Physics.

[67]  Wlodzimierz Bryc,et al.  A remark on the connection between the large deviation principle and the central limit theorem , 1993 .