Geometry and Dynamics for Markov Chain Monte Carlo

Markov chain Monte Carlo methods have revolutionized mathematical computation and enabled statistical inference within many previously intractable models. In this context, Hamiltonian dynamics have been proposed as an efficient way of building chains that can explore probability densities efficiently. The method emerges from physics and geometry, and these links have been extensively studied over the past thirty years. The aim of this review is to provide a comprehensive introduction to the geometric tools used in Hamiltonian Monte Carlo at a level accessible to statisticians, machine learners, and other users of the methodology with only a basic understanding of Monte Carlo methods. This will be complemented with some discussion of the most recent advances in the field, which we believe will become increasingly relevant to scientists.

[1]  Andrew M. Stuart,et al.  Geometric MCMC for infinite-dimensional inverse problems , 2016, J. Comput. Phys..

[2]  George Casella,et al.  A Short History of Markov Chain Monte Carlo: Subjective Recollections from Incomplete Data , 2008, 0808.2902.

[3]  Scott S. Hampton,et al.  Shadow hybrid Monte Carlo: an efficient propagator in phase space of macromolecules , 2004 .

[4]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[5]  Nando de Freitas,et al.  Adaptive Hamiltonian and Riemann manifold Monte Carlo samplers , 2013, ICML 2013.

[6]  Scott S. Hampton,et al.  A separable shadow Hamiltonian hybrid Monte Carlo method. , 2009, The Journal of chemical physics.

[7]  Lawrence Carin,et al.  On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators , 2015, NIPS.

[8]  J. M. Sanz-Serna,et al.  Optimal tuning of the hybrid Monte Carlo algorithm , 2010, 1001.4460.

[9]  H. Yoshida.  Construction of higher order symplectic integrators , 1990 .

[10]  Tianqi Chen,et al.  A Complete Recipe for Stochastic Gradient MCMC , 2015, NIPS.

[11]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[12]  Yee Whye Teh,et al.  Bayesian Learning via Stochastic Gradient Langevin Dynamics , 2011, ICML.

[13]  D. Stoyan,et al.  Stochastic Geometry and Its Applications , 1989 .

[14]  Mark A. Girolami,et al.  Emulation of higher-order tensors in manifold Monte Carlo methods for Bayesian Inverse Problems , 2015, J. Comput. Phys..

[15]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[16]  M. Girolami,et al.  Solving large-scale PDE-constrained Bayesian inverse problems with Riemann manifold Hamiltonian Monte Carlo , 2014, 1407.1517.

[17]  David Dunson,et al.  Geometrically Tempered Hamiltonian Monte Carlo , 2016, 1604.00872.

[18]  B. Berne,et al.  Novel methods of sampling phase space in the simulation of biological systems. , 1997, Current opinion in structural biology.

[19]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[20]  Tijana Radivojević,et al.  Constant pressure hybrid Monte Carlo simulations in GROMACS , 2014, Journal of Molecular Modeling.

[21]  K. Zygalakis,et al.  (Non-) asymptotic properties of Stochastic Gradient Langevin Dynamics , 2015, 1501.00438.

[22]  Ajay,et al.  Can we learn to distinguish between "drug-like" and "nondrug-like" molecules? , 1998, Journal of medicinal chemistry.

[23]  Ian L. Dryden,et al.  Geometry driven statistics , 2015 .

[24]  Radford M. Neal.  MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[25]  Assyr Abdulle,et al.  Long Time Accuracy of Lie-Trotter Splitting Methods for Langevin Dynamics , 2015, SIAM J. Numer. Anal..

[26]  Mrinal K. Sen,et al.  Transdimensional seismic inversion using the reversible jump Hamiltonian Monte Carlo algorithm , 2017 .

[27]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[28]  J. Rosenthal,et al.  Optimal scaling of discrete approximations to Langevin diffusions , 1998 .

[29]  Michael Betancourt,et al.  A Conceptual Introduction to Hamiltonian Monte Carlo , 2017, 1701.02434.

[30]  Radford M. Neal,et al.  ANALYSIS OF A NONREVERSIBLE MARKOV CHAIN SAMPLER , 2000 .

[31]  Tianqi Chen,et al.  Stochastic Gradient Hamiltonian Monte Carlo , 2014, ICML.

[32]  Yee Whye Teh,et al.  Consistency and Fluctuations For Stochastic Gradient Langevin Dynamics , 2014, J. Mach. Learn. Res..

[33]  Paul Marriott,et al.  Preferred Point Geometry and Statistical Manifolds , 1993 .

[34]  Jesús María Sanz-Serna,et al.  Extra Chance Generalized Hybrid Monte Carlo , 2014, J. Comput. Phys..

[35]  M. Girolami,et al.  Markov Chain Monte Carlo from Lagrangian Dynamics , 2015, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[36]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[37]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[38]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[39]  Paolo Rossi,et al.  A Comparison of Numerical Algorithms for Dynamical Fermions , 1990 .

[40]  G. Roberts,et al.  MCMC Methods for Functions: ModifyingOld Algorithms to Make Them Faster , 2012, 1202.0709.

[41]  J. M. Sanz-Serna,et al.  Hybrid Monte Carlo on Hilbert spaces , 2011 .

[42]  M. T. Vaughn.  Geometry in Physics , 2008 .

[43]  M. Girolami,et al.  Geodesic Monte Carlo on Embedded Manifolds , 2013, Scandinavian journal of statistics, theory and applications.

[44]  V. Arnold.  Mathematical Methods of Classical Mechanics , 1974 .

[45]  Sugar,et al.  New algorithm for the numerical simulation of fermions. , 1986, Physical review. B, Condensed matter.

[46]  M. Betancourt,et al.  On the geometric ergodicity of Hamiltonian Monte Carlo , 2016, Bernoulli.

[47]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[48]  A. Horowitz.  A generalized guided Monte Carlo algorithm , 1991 .

[49]  Benedict Leimkuhler,et al.  Simulating Hamiltonian Dynamics: Acknowledgements , 2005 .

[50]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[51]  J. M. Sanz-Serna,et al.  Compressible generalized hybrid Monte Carlo. , 2014, The Journal of chemical physics.

[52]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[53]  R. Kass,et al.  Geometrical Foundations of Asymptotic Inference , 1997 .