A Condition Number for Hamiltonian Monte Carlo.

Hamiltonian Monte Carlo is a popular sampling technique for smooth target densities. The scale lengths of the target have long been believed to influence sampling efficiency, but quantitative measures intrinsic to the target have been lacking. In this paper, we restrict attention to multivariate Gaussian targets, and obtain a condition number corresponding to sampling efficiency. This number, based on a mix of spectral and Schatten norms, quantifies the number of leapfrog steps needed to efficiently sample. We demonstrate its utility by using the condition number to analyze preconditioning techniques for HMC.

[1]  Joel Nothman,et al.  SciPy 1.0-Fundamental Algorithms for Scientific Computing in Python , 2019, ArXiv.

[2]  Aki Vehtari,et al.  Selecting the Metric in Hamiltonian Monte Carlo , 2019, 1905.11916.

[3]  Joshua V. Dillon,et al.  NeuTra-lizing Bad Geometry in Hamiltonian Monte Carlo Using Neural Transport , 2019, 1903.03704.

[4]  Benjamin Peherstorfer,et al.  A transport-based multifidelity preconditioner for Markov chain Monte Carlo , 2018, Advances in Computational Mathematics.

[5]  Y. Marzouk,et al.  Transport Map Accelerated Markov Chain Monte Carlo , 2014, SIAM/ASA J. Uncertain. Quantification.

[6]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[7]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[8]  Michael Betancourt,et al.  A Conceptual Introduction to Hamiltonian Monte Carlo , 2017, 1701.02434.

[9]  Nikolaos V. Sahinidis,et al.  Simulation optimization: a review of algorithms and applications , 2014, 4OR.

[10]  M. Betancourt,et al.  Optimizing The Integrator Step Size for Hamiltonian Monte Carlo , 2014, 1411.6669.

[11]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[12]  Radford M. Neal.  MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[13]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[14]  A. Bose,et al.  Limiting Spectral Distribution of , 2011 .

[15]  Arup Bose,et al.  Limiting spectral distribution of XX' matrices , 2010 .

[16]  J. M. Sanz-Serna,et al.  Optimal tuning of the hybrid Monte Carlo algorithm , 2010, 1001.4460.

[17]  Arup Bose,et al.  Another look at the moment method for large dimensional random matrices , 2008 .

[18]  Christophe Andrieu,et al.  A tutorial on adaptive MCMC , 2008, Stat. Comput..

[19]  Z. Bai,et al.  METHODOLOGIES IN SPECTRAL ANALYSIS OF LARGE DIMENSIONAL RANDOM MATRICES, A REVIEW , 2008 .

[20]  E. Hairer,et al.  Simulating Hamiltonian dynamics , 2006, Math. Comput..

[21]  Per Christian Hansen,et al.  Computation of the singular value expansion , 1988, Computing.

[22]  B. Leimkuhler,et al.  Simulating Hamiltonian Dynamics , 2005 .

[23]  E. Somersalo,et al.  Statistical and computational inverse problems , 2004 .

[24]  Arvind Sharma,et al.  A Conceptual Introduction , 2001 .

[25]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[26]  Radford M. Neal.  Bayesian learning for neural networks , 1995 .

[27]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[28]  R. Durrett.  Probability: Theory and Examples , 1993 .

[29]  Paul B. Mackenze.  An Improved Hybrid Monte Carlo Method , 1989 .

[30]  A. Kennedy,et al.  Hybrid Monte Carlo , 1988 .

[31]  J. W. Silverstein.  The Smallest Eigenvalue of a Large Dimensional Wishart Matrix , 1985 .

[32]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .