Markov chain Monte Carlo algorithms with sequential proposals

We explore a general framework in Markov chain Monte Carlo (MCMC) sampling where sequential proposals are tried as a candidate for the next state of the Markov chain. This sequential-proposal framework can be applied to various existing MCMC methods, including Metropolis–Hastings algorithms using random proposals and methods that use deterministic proposals such as Hamiltonian Monte Carlo (HMC) or the bouncy particle sampler. Sequential-proposal MCMC methods construct the same Markov chains as those constructed by the delayed rejection method under certain circumstances. In the context of HMC, the sequential-proposal approach has been proposed as extra chance generalized hybrid Monte Carlo (XCGHMC). We develop two novel methods in which the trajectories leading to proposals in HMC are automatically tuned to avoid doubling back, as in the No-U-Turn sampler (NUTS). The numerical efficiency of these new methods compare favorably to the NUTS. We additionally show that the sequential-proposal bouncy particle sampler enables the constructed Markov chain to pass through regions of low target density and thus facilitates better mixing of the chain when the target density is multimodal.

[1]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[2]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[3]  J. Liouville,et al.  Note sur la Théorie de la Variation des constantes arbitraires. , 1838 .

[4]  J. Rosenthal,et al.  On adaptive Markov chain Monte Carlo algorithms , 2005 .

[5]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[6]  A. Mira.  On Metropolis-Hastings algorithms with delayed rejection , 2001 .

[7]  J. Rosenthal,et al.  Optimal scaling of discrete approximations to Langevin diffusions , 1998 .

[8]  M. Plummer,et al.  CODA: convergence diagnosis and output analysis for MCMC , 2006 .

[9]  J. M. Sanz-Serna,et al.  Optimal tuning of the hybrid Monte Carlo algorithm , 2010, 1001.4460.

[10]  J. Rosenthal,et al.  Convergence of Slice Sampler Markov Chains , 1999 .

[11]  Radford M. Neal.  MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[12]  J. Rosenthal,et al.  Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms , 2007, Journal of Applied Probability.

[13]  Sourendu Gupta,et al.  The acceptance probability in the hybrid Monte Carlo method , 1990 .

[14]  S. Kou,et al.  Equi-energy sampler with applications in statistical inference and statistical mechanics , 2005, math/0507080.

[15]  Jeffrey S. Rosenthal,et al.  Optimal Proposal Distributions and Adaptive MCMC , 2011 .

[16]  G. Fort,et al.  Limit theorems for some adaptive MCMC algorithms with subgeometric kernels , 2008, 0807.2952.

[17]  P. Green,et al.  Delayed rejection in reversible jump Metropolis–Hastings , 2001 .

[18]  L Tierney,et al.  Some adaptive monte carlo methods for Bayesian inference. , 1999, Statistics in medicine.

[19]  Heikki Haario,et al.  Componentwise adaptation for high dimensional MCMC , 2005, Comput. Stat..

[20]  P. Fearnhead,et al.  The Random Walk Metropolis: Linking Theory and Practice Through a Case Study , 2010, 1011.6217.

[21]  Ernst Hairer,et al.  Simulating Hamiltonian dynamics , 2006, Math. Comput..

[22]  C. Andrieu,et al.  On the ergodicity properties of some adaptive MCMC algorithms , 2006, math/0610317.

[23]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[24]  P. Peskun,et al.  Optimum Monte-Carlo sampling using Markov chains , 1973 .

[25]  J. Sexton,et al.  Hamiltonian evolution for the hybrid Monte Carlo algorithm , 1992 .

[26]  Jun S. Liu,et al.  The Multiple-Try Method and Local Optimization in Metropolis Sampling , 2000 .

[27]  S. Walker.  Invited comment on the paper "Slice Sampling" by Radford Neal , 2003 .

[28]  Radford M. Neal.  Slice Sampling , 2003, The Annals of Statistics.

[29]  C. Andrieu,et al.  Peskun-Tierney ordering for Markov chain and process Monte Carlo: beyond the reversible scenario , 2019, 1906.06197.

[30]  J. M. Sanz-Serna,et al.  Compressible generalized hybrid Monte Carlo. , 2014, The Journal of chemical physics.

[31]  Radford M. Neal.  An improved acceptance procedure for the hybrid Monte Carlo algorithm , 1992, hep-lat/9208011.

[32]  A. Gelman,et al.  Weak convergence and optimal scaling of random walk Metropolis algorithms , 1997 .

[33]  Michael Betancourt,et al.  A Conceptual Introduction to Hamiltonian Monte Carlo , 2017, 1701.02434.

[34]  A. Horowitz.  A generalized guided Monte Carlo algorithm , 1991 .

[35]  G. Roberts,et al.  Perfect slice samplers , 2001 .

[36]  A. Doucet,et al.  Piecewise-Deterministic Markov Chain Monte Carlo , 2017, 1707.05296.

[37]  C. Andrieu,et al.  On the efficiency of adaptive MCMC algorithms , 2007 .

[38]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[39]  G. Parisi,et al.  Simulated tempering: a new Monte Carlo scheme , 1992, hep-lat/9205018.

[40]  Jesús María Sanz-Serna,et al.  Extra Chance Generalized Hybrid Monte Carlo , 2014, J. Comput. Phys..

[41]  J. Sexton,et al.  Hamiltonian evolution for the hybrid Monte Carlo algorithm , 1992 .

[42]  Christophe Andrieu,et al.  A tutorial on adaptive MCMC , 2008, Stat. Comput..

[43]  J. S. Rosenthal.  Optimal scaling of discrete approximations to Langevin , 1997 .

[44]  C. Geyer.  Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[45]  E A J F Peters,et al.  Rejection-free Monte Carlo sampling for general potentials. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[46]  Jonathan R Goodman,et al.  Ensemble samplers with affine invariance , 2010 .

[47]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[48]  L. Tierney.  A note on Metropolis-Hastings kernels for general state spaces , 1998 .

[49]  A. Doucet,et al.  The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method , 2015, 1510.02451.

[50]  Ben Calderhead,et al.  A general construction for parallelizing Metropolis−Hastings algorithms , 2014, Proceedings of the National Academy of Sciences.

[51]  K. Hukushima,et al.  Exchange Monte Carlo Method and Application to Spin Glass Simulations , 1995, cond-mat/9512035.

[52]  Jascha Sohl-Dickstein,et al.  Hamiltonian Monte Carlo Without Detailed Balance , 2014, ICML.

[53]  S. Duane,et al.  Hybrid Monte Carlo , 1987 .