Honest Exploration of Intractable Probability Distributions via Markov Chain Monte Carlo

Two important questions that must be answered whenever a Markov chain Monte Carlo (MCMC) algorithm is used are (Q1) What is an appropriate burn-in? and (Q2) How long should the sampling continue after burn-in? Developing rigorous answers to these questions presently requires a detailed study of the convergence properties of the underlying Markov chain. Consequently, in most practical applications of MCMC, exact answers to (Q1) and (Q2) are not sought. The goal of this paper is to demystify the analysis that leads to honest answers to (Q1) and (Q2). The authors hope that this article will serve as a bridge between those developing Markov chain theory and practitioners using MCMC to solve practical problems. The ability to address (Q1) and (Q2) formally comes from establishing a drift condition and an associated minorization condition, which together imply that the underlying Markov chain is geometrically ergodic. In this article, we explain exactly what drift and minorization are as well as how and why these conditions can be used to form rigorous answers to (Q1) and (Q2). The basic ideas are as follows. The results of Rosenthal (1995) and Roberts and Tweedie (1999) allow one to use drift and minorization conditions to construct a formula giving an analytic upper bound on the distance to stationarity. A rigorous answer to (Q1) can be calculated using this formula. The desired characteristics of the target distribution are typically estimated using ergodic averages. Geometric ergodicity of the underlying Markov chain implies that there are central limit theorems available for ergodic averages (Chan and Geyer 1994). The regenerative simulation technique (Mykland, Tierney and Yu, 1995; Robert, 1995) can be used to get a consistent estimate of the variance of the asymptotic normal distribution. Hence, an asymptotic standard error can be calculated, which provides an answer to (Q2) in the sense that an appropriate time to stop sampling can be determined. The methods are illustrated using a Gibbs sampler for a Bayesian version of the one-way random effects model and a data set concerning styrene exposure.

[1]  Michael A. Crane,et al.  Simulating Stable Stochastic Systems: III. Regenerative Processes and Discrete-Event Simulations , 1975, Oper. Res..

[2]  E. Nummelin,et al.  A splitting technique for Harris recurrent Markov chains , 1978 .

[3]  K. Athreya,et al.  A New Approach to the Limit Theory of Recurrent Markov Chains , 1978 .

[4]  Linus Schrage,et al.  A guide to simulation , 1983 .

[5]  Paul Bratley,et al.  A guide to simulation , 1983 .

[6]  E. Nummelin.  General irreducible Markov chains and non-negative operators: Notes and comments , 1984 .

[7]  P. Glynn.  Regenerative structure of Markov chains simulated via common random numbers , 1985 .

[8]  Brian D. Ripley,et al.  Stochastic Simulation , 2005 .

[9]  Donald L. Iglehart,et al.  Simulation Output Analysis Using Standardized Time Series , 1990, Math. Oper. Res..

[10]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[11]  S. E. Hills,et al.  Illustration of Bayesian Inference in Normal Data Models Using Gibbs Sampling , 1990 .

[12]  P. Diaconis,et al.  Geometric Bounds for Eigenvalues of Markov Chains , 1991 .

[13]  T. Lindvall.  Lectures on the Coupling Method , 1992 .

[14]  C. Hwang,et al.  Convergence rates of the Gibbs sampler, the Metropolis algorithm and other single-site updating dynamics , 1993 .

[15]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[16]  S. Meyn,et al.  Computable Bounds for Geometric Convergence Rates of Markov Chains , 1994 .

[17]  J. Rosenthal.  Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[18]  Jun S. Liu,et al.  Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes , 1994 .

[19]  S. Ingrassia.  ON THE RATE OF CONVERGENCE OF THE METROPOLIS ALGORITHM AND GIBBS SAMPLER BY GEOMETRIC BOUNDS , 1994 .

[20]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[21]  Bin Yu,et al.  Regeneration in Markov chain samplers , 1995 .

[22]  C. Robert.  Convergence Control Methods for Markov Chain Monte Carlo Algorithms , 1995 .

[23]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[24]  R. Tweedie,et al.  Rates of convergence of the Hastings and Metropolis algorithms , 1996 .

[25]  Jeffrey S. Rosenthal,et al.  Analysis of the Gibbs Sampler for a Model Related to James-stein Estimators , 2007 .

[26]  R. Tweedie,et al.  Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms , 1996 .

[27]  Richard L. Tweedie,et al.  Geometric Convergence Rates for Stochastically Ordered Markov Chains , 1996, Math. Oper. Res..

[28]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[29]  Robert H. Lyles,et al.  Assessing Regulatory Compliance of Occupational Exposures Via the Balanced One-Way Random Effects ANOVA Model , 1997 .

[30]  C. McCulloch.  Maximum Likelihood Algorithms for Generalized Linear Mixed Models , 1997 .

[31]  L. Tierney.  A note on Metropolis-Hastings kernels for general state spaces , 1998 .

[32]  Ranjini Natarajan,et al.  Gibbs Sampling with Diffuse Proper Priors: A Valid Approach to Data-Driven Inference? , 1998 .

[33]  Mary Kathryn Cowles,et al.  A simulation approach to convergence rates for Markov chain Monte Carlo algorithms , 1998, Stat. Comput..

[34]  P. Diaconis,et al.  Algebraic algorithms for sampling from conditional distributions , 1998 .

[35]  J. Rosenthal,et al.  On convergence rates of Gibbs samplers for uniform distributions , 1998 .

[36]  C. Geyer,et al.  Geometric Ergodicity of Gibbs and Block Gibbs Samplers for a Hierarchical Random Effects Model , 1998 .

[37]  G. Roberts,et al.  Adaptive Markov Chain Monte Carlo through Regeneration , 1998 .

[38]  Gareth O. Roberts,et al.  Markov‐chain monte carlo: Some practical implications of theoretical results , 1998 .

[39]  G. O. Roberts.  A note on acceptance rate criteria for CLTS for Metropolis–Hastings algorithms , 1999 .

[40]  R. Tweedie,et al.  Bounds on regeneration times and convergence rates for Markov chains fn1 fn1 Work supported in part , 1999 .

[41]  Wai Kong Yuen,et al.  Applications of geometric bounds to the convergence rate of Markov chains on Rn , 2000 .

[42]  S. F. Jarner,et al.  Geometric ergodicity of Metropolis algorithms , 2000 .

[43]  George Casella,et al.  Implementations of the Monte Carlo EM Algorithm , 2001 .

[44]  G. Roberts,et al.  Polynomial convergence rates of Markov chains. , 2002 .

[45]  S. Rosenthal,et al.  A review of asymptotic convergence for general state space Markov chains , 2002 .

[46]  Galin L. Jones,et al.  On the applicability of regenerative simulation in Markov chain Monte Carlo , 2002 .