Peskun-Tierney ordering for Markov chain and process Monte Carlo: beyond the reversible scenario

Historically time-reversibility of the transitions or processes underpinning Markov chain Monte Carlo methods (MCMC) has played a key r\^ole in their development, while the self-adjointness of associated operators together with the use of classical functional analysis techniques on Hilbert spaces have led to powerful and practically successful tools to characterize and compare their performance. Similar results for algorithms relying on nonreversible Markov processes are scarce. We show that for a type of nonreversible Monte Carlo Markov chains and processes, of current or renewed interest in the Physics and Statistical literatures, it is possible to develop comparison results which closely mirror those available in the reversible scenario. We show that these results shed light on earlier literature, proving some conjectures and strengthening some earlier results.

[1]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[2]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[3]  P. Peskun,et al.  Optimum Monte-Carlo sampling using Markov chains , 1973 .

[4]  J. Voigt.  On the perturbation theory for strongly continuous semigroups , 1977 .

[5]  N. Maigret.  Théorème de limite centrale fonctionnel pour une chaîne de Markov récurrente au sens de Harris et positive , 1978 .

[6]  A. Kennedy,et al.  Hybrid Monte Carlo , 1988 .

[7]  S. Caracciolo,et al.  Nonlocal Monte Carlo algorithm for self-avoiding walks with fixed endpoints , 1990 .

[8]  A. Horowitz.  A generalized guided Monte Carlo algorithm , 1991 .

[9]  M.H.A. Davis,et al.  Markov Models & Optimization , 1993 .

[10]  Jun S. Liu.  Peskun's theorem and a modified discrete-state Gibbs sampler , 1996 .

[11]  Sean P. Meyn,et al.  A Liapounov bound for solutions of the Poisson equation , 1996 .

[12]  Paul Gustafson,et al.  A guided walk Metropolis algorithm , 1998, Stat. Comput..

[13]  J. Lamb,et al.  Time-reversal symmetry in dynamical systems: a survey , 1998 .

[14]  L. Tierney.  A note on Metropolis-Hastings kernels for general state spaces , 1998 .

[15]  W. Wefelmeyer,et al.  Information bounds for Gibbs samplers , 1998 .

[16]  D. Ceperley,et al.  The penalty method for random walks with uncertain energies , 1998, physics/9812035.

[17]  Radford M. Neal,et al.  ANALYSIS OF A NONREVERSIBLE MARKOV CHAIN SAMPLER , 2000 .

[18]  Antonietta Mira,et al.  Ordering and Improving the Performance of Monte Carlo Markov Chains , 2001 .

[19]  Radford M. Neal.  Improving Asymptotic Variance of MCMC Estimators: Non-reversible Chains are Better , 2004, math/0407281.

[20]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[21]  Michael L. Overton,et al.  Optimizing the asymptotic convergence rate of the Diaconis-Holmes-Neal sampler , 2007, Adv. Appl. Math..

[22]  Michael Chertkov,et al.  Irreversible Monte Carlo Algorithms for Efficient Sampling , 2008, ArXiv.

[23]  Antonietta Mira,et al.  AN EXTENSION OF PESKUN AND TIERNEY ORDERINGS TO CONTINUOUS TIME MARKOV CHAINS , 2008 .

[24]  J. Hobert,et al.  A theoretical comparison of the data augmentation, marginal augmentation and PX-DA algorithms , 2008, 0804.0671.

[25]  A. Faggionato,et al.  Non-equilibrium Thermodynamics of Piecewise Deterministic Markov Processes , 2009 .

[26]  G. Roberts,et al.  CLTs and Asymptotic Variance of Time-Sampled Markov Chains , 2011, 1102.2171.

[27]  A. Doucet,et al.  Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator , 2012, 1210.1871.

[28]  E A J F Peters,et al.  Rejection-free Monte Carlo sampling for general potentials. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[29]  N. Pillai,et al.  A Function Space HMC Algorithm With Second Order Langevin Diffusion Limit , 2013, 1308.0543.

[30]  K. Hukushima,et al.  An irreversible Markov-chain Monte Carlo method with skew detailed balance conditions , 2013 .

[31]  R. Douc,et al.  Comparison of asymptotic variances of inhomogeneous Markov chains with application to Markov chain Monte Carlo methods , 2013, 1307.3719.

[32]  J. M. Sanz-Serna,et al.  Compressible generalized hybrid Monte Carlo. , 2014, The Journal of chemical physics.

[33]  Jascha Sohl-Dickstein,et al.  Hamiltonian Monte Carlo Without Detailed Balance , 2014, ICML.

[34]  Ryan O'Donnell,et al.  Analysis of Boolean Functions , 2014, ArXiv.

[35]  Marija Vucelja.  Lifting -- A nonreversible Markov chain Monte Carlo Algorithm , 2014 .

[36]  Gareth O. Roberts,et al.  Minimising MCMC variance via diffusion limits, with an application to simulated tempering , 2014 .

[37]  Christophe Andrieu,et al.  Establishing some order amongst exact approximations of MCMCs , 2014, 1404.6909.

[38]  J. Rosenthal,et al.  Surprising Convergence Properties of Some Simple Gibbs Samplers under Various Scans , 2015 .

[39]  Jesús María Sanz-Serna,et al.  Extra Chance Generalized Hybrid Monte Carlo , 2014, J. Comput. Phys..

[40]  Spectral Bounds for Certain Two-Factor Non-Reversible MCMC Algorithms , 2015 .

[41]  G. Roberts,et al.  A piecewise deterministic scaling limit of Lifted Metropolis-Hastings in the Curie-Weiss model , 2015, 1509.00302.

[42]  C. Andrieu,et al.  Convergence properties of pseudo-marginal Markov chain Monte Carlo algorithms , 2012, 1210.1484.

[43]  J. M. Sanz-Serna,et al.  Randomized Hamiltonian Monte Carlo , 2015, 1511.09382.

[44]  A. Doucet,et al.  The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method , 2015, 1510.02451.

[45]  Koji Hukushima,et al.  Eigenvalue analysis of an irreversible random walk with skew detailed balance conditions. , 2015, Physical review. E.

[46]  C. Andrieu.  On random- and systematic-scan samplers , 2016 .

[47]  Michela Ottobre,et al.  Markov Chain Monte Carlo and Irreversibility , 2016 .

[48]  Konstantinos Spiliopoulos,et al.  Improving the Convergence of Reversible Samplers , 2016 .

[49]  Anthony Lee,et al.  Pseudo‐marginal Metropolis‐Hastings sampling using averages of unbiased estimators , 2017 .

[50]  A. Duncan,et al.  Limit theorems for the zig-zag process , 2016, Advances in Applied Probability.

[51]  Aaron Smith,et al.  The use of a single pseudo-sample in approximate Bayesian computation , 2014, Stat. Comput..

[52]  Generalized and hybrid Metropolis-Hastings overdamped Langevin algorithms , 2017, 1701.05833.

[53]  Paul Fearnhead,et al.  Piecewise Deterministic Markov Processes for Continuous-Time Monte Carlo , 2016, Statistical Science.

[54]  P. Fearnhead,et al.  The Zig-Zag process and super-efficient sampling for Bayesian analysis of big data , 2016, The Annals of Statistics.

[55]  S. Lunel,et al.  Spectral analysis of the zigzag process , 2019, Annales de l'Institut Henri Poincaré, Probabilités et Statistiques.

[56]  Alain Durmus,et al.  Piecewise deterministic Markov processes and their invariant measures , 2018, Annales de l'Institut Henri Poincaré, Probabilités et Statistiques.

[57]  Christophe Andrieu,et al.  Hypocoercivity of piecewise deterministic Markov process-Monte Carlo , 2018, The Annals of Applied Probability.