Markov Chain Monte Carlo conver-gence diagnostics: a comparative review

Abstract A critical issue for users of Markov chain Monte Carlo (MCMC) methods in applications is how to determine when it is safe to stop sampling and use the samples to estimate characteristics of the distribution of interest. Research into methods of computing theoretical convergence bounds holds promise for the future but to date has yielded relatively little of practical use in applied work. Consequently, most MCMC users address the convergence problem by applying diagnostic tools to the output produced by running their samplers. After giving a brief overview of the area, we provide an expository review of 13 convergence diagnostics, describing the theoretical basis and practical implementation of each. We then compare their performance in two simple models and conclude that all of the methods can fail to detect the sorts of convergence failure that they were designed to identify. We thus recommend a combination of strategies aimed at evaluating and accelerating MCMC sampler convergence, including ap...

[1]  Bin Yu,et al.  Regeneration in Markov chain samplers , 1995 .

[2]  J. Q. Smith,et al.  1. Bayesian Statistics 4 , 1993 .

[3]  A. Gelfand,et al.  Efficient parametrisations for normal linear mixed models , 1995 .

[4]  Jim Albert.  Teaching Bayesian Statistics Using Sampling Methods and MINITAB , 1993 .

[5]  Philip Heidelberger,et al.  Simulation Run Length Control in the Presence of an Initial Transient , 1983, Oper. Res..

[6]  James O. Berger,et al.  Bayesian Estimation of Fuel Economy Potential Due to Technology Improvements , 1993 .

[7]  Lee W. Schruben,et al.  Detecting Initialization Bias in Simulation Output , 1982, Oper. Res..

[8]  G. Casella,et al.  Explaining the Gibbs Sampler , 1992 .

[9]  A. Raftery,et al.  How Many Iterations in the Gibbs Sampler , 1991 .

[10]  V. Johnson.  Studying Convergence of Markov Chain Monte Carlo Algorithms Using Coupled Sample Paths , 1996 .

[11]  Walter R. Gilks,et al.  BUGS - Bayesian inference Using Gibbs Sampling Version 0.50 , 1995 .

[12]  M. Tanner,et al.  Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler , 1992 .

[13]  Joong-Kweon Sohn,et al.  Convergence Diagnostics for the Gibbs Sampler , 1996 .

[14]  N. Best,et al.  Convergence Diagnosis and Output Analysis Software for Gibbs sampling output Version 0.30 , 1995 .

[15]  Brian D. Ripley,et al.  Stochastic Simulation , 2005 .

[16]  J. Rosenthal.  Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[17]  J. Rosenthal.  RATES OF CONVERGENCE FOR GIBBS SAMPLING FOR VARIANCE COMPONENT MODELS , 1995 .

[18]  A. Gelfand,et al.  Maximum-likelihood estimation for constrained- or missing-data models , 1993 .

[19]  C. Buck,et al.  Detecting a Change in the Shape of a Prehistoric Corbelled Tomb , 1993 .

[20]  L. Mark Berliner,et al.  Subsampling the Gibbs Sampler , 1994 .

[21]  J. Besag,et al.  Spatial Statistics and Bayesian Computation , 1993 .

[22]  Bradley P. Carlin,et al.  Hierarchical Bayes Models for the Progression of HIV Infection Using Longitudinal CD4+ Counts. , 1992 .

[23]  Hani Doss.  Discussion: Markov Chains for Exploring Posterior Distributions , 1994 .

[24]  Peter Green,et al.  Spatial statistics and Bayesian computation (with discussion) , 1993 .

[25]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[26]  Lee W. Schruben,et al.  Optimal Tests for Initialization Bias in Simulation Output , 1983, Oper. Res..

[27]  A. Gelfand,et al.  Hierarchical Bayes Models for the Progression of HIV Infection Using Longitudinal CD4 T-Cell Numbers , 1992 .

[28]  Jim Albert,et al.  A Bayesian Analysis of a Poisson Random Effects Model for Home Run Hitters , 1992 .

[29]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[30]  W. Cleveland.  Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[31]  John Geweke,et al.  Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments , 1991 .

[32]  L. Tierney.  Rejoinder: Markov Chains for Exploring Posterior Distributions , 1994 .

[33]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[34]  Jeffrey S. Rosenthal,et al.  Analysis of the Gibbs Sampler for a Model Related to James-stein Estimators , 2007 .

[35]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[36]  Walter R. Gilks,et al.  A Language and Program for Complex Bayesian Modelling , 1994 .

[37]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[38]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[39]  B. Carlin,et al.  On the Convergence of Successive Substitution Sampling , 1992 .

[40]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[41]  A. Gelfand,et al.  On Markov Chain Monte Carlo Acceleration , 1994 .

[42]  Wang,et al.  Nonuniversal critical dynamics in Monte Carlo simulations. , 1987, Physical review letters.

[43]  J. Rosenthal.  Rates of Convergence for Data Augmentation on Finite Sample Spaces , 1993 .

[44]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Donald B. Rubin,et al.  [Practical Markov Chain Monte Carlo]: Rejoinder: Replication without Contrition , 1992 .

[46]  A. Zellner,et al.  Gibbs Sampler Convergence Criteria , 1995 .

[47]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[48]  Martin A. Tanner,et al.  Posterior Computations for Censored Regression Data , 1990 .

[49]  K. Cowles,et al.  CODA: convergence diagnosis and output analysis software for Gibbs sampling output , 1995 .

[50]  M. Tanner,et al.  [Practical Markov Chain Monte Carlo]: Comment: Monitoring Convergence of the Gibbs Sampler: Further Experience with the Gibbs Stopper , 1992 .

[51]  C. Geyer,et al.  Discussion: Markov Chains for Exploring Posterior Distributions , 1994 .