Practical Markov Chain Monte Carlo

Markov chain Monte Carlo using the Metropolis-Hastings algorithm is a general method for the simulation of stochastic processes having probability densities known up to a constant of proportionality. Despite recent advances in its theory, the practice has remained controversial. This article makes the case for basing all inference on one long run of the Markov chain and estimating the Monte Carlo error by standard nonparametric methods well-known in the time-series and operations research literature. In passing it touches on the Kipnis-Varadhan central limit theorem for reversible Markov chains, on some new variance estimators, on judging the relative efficiency of competing Monte Carlo schemes, on methods for constructing more rapidly mixing Markov chains and on diagnostics for Markov chain Monte Carlo.

[1]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[2]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[3]  G. S. Fishman Principles of Discrete Event Simulation , 1978 .

[4]  Y. Ogata,et al.  Estimation of interaction potentials of spatial point patterns through the maximum likelihood procedure , 1981 .

[5]  Bruce W. Schmeiser,et al.  Batch Size Effects in the Analysis of Simulation Output , 1982, Oper. Res..

[6]  Lee W. Schruben,et al.  Confidence Interval Estimation Using Standardized Time Series , 1983, Oper. Res..

[7]  D. B. Preston Spectral Analysis and Time Series , 1983 .

[8]  Y. Ogata,et al.  Likelihood Analysis of Spatial Point Patterns , 1984 .

[9]  E. Nummelin General irreducible Markov chains and non-negative operators: Notes and comments , 1984 .

[10]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Averill M. Law,et al.  An Analytical Evaluation of Alternative Strategies in Steady-State Simulation , 1984, Oper. Res..

[12]  S. Varadhan,et al.  Central limit theorem for additive functionals of reversible Markov processes and applications to simple exclusions , 1986 .

[13]  Reuven Y. Rubinstein,et al.  Monte Carlo Optimization, Simulation and Sensitivity of Queueing Networks , 1986 .

[14]  J. Besag On the Statistical Analysis of Dirty Pictures , 1986 .

[15]  B. Tóth Persistent random walks in random environment , 1986 .

[16]  Wang,et al.  Nonuniversal critical dynamics in Monte Carlo simulations. , 1987, Physical review letters.

[17]  Adrian F. M. Smith,et al.  Bayesian computation via the gibbs sampler and related markov chain monte carlo methods (with discus , 1993 .

[18]  L. Younes Estimation and annealing for Gibbsian fields , 1988 .

[19]  N. Mendell,et al.  Probabilistic Measures of Adequacy of a Numerical Search for a Global Maximum , 1989 .

[20]  J. Besag,et al.  Generalized Monte Carlo significance tests , 1989 .

[21]  Y. Ogata,et al.  Likelihood estimation of soft-core interaction potentials for Gibbsian point patterns , 1989 .

[22]  Donald L. Iglehart,et al.  Simulation Output Analysis Using Standardized Time Series , 1990, Math. Oper. Res..

[23]  B. D. Riley,et al.  Iterative simulation methods , 1990 .

[24]  R. Swendsen,et al.  Cluster Monte Carlo algorithms , 1990 .

[25]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[26]  C. Geyer Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[27]  A. Raftery,et al.  How Many Iterations in the Gibbs Sampler , 1991 .

[28]  John Geweke,et al.  Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments , 1991 .

[29]  J. Besag,et al.  Bayesian image restoration, with two applications in spatial statistics , 1991 .

[30]  J. Besag,et al.  Sequential Monte Carlo p-values , 1991 .

[31]  Y. Amit,et al.  A Nonhomogeneous Markov Process for the Estimation of Gaussian Random Fields with Nonlinear Observations , 1991 .

[32]  P. Diaconis,et al.  Geometric Bounds for Eigenvalues of Markov Chains , 1991 .

[33]  Y. Amit On rates of convergence of stochastic relaxation for Gaussian and non-Gaussian distributions , 1991 .

[34]  E A Thompson,et al.  Evaluation of likelihood ratios for complex genetic models. , 1991, IMA journal of mathematics applied in medicine and biology.

[35]  P. Green,et al.  Metropolis Methods, Gaussian Proposals and Antithetic Variables , 1992 .

[36]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[37]  C. Geyer,et al.  Constrained Monte Carlo Maximum Likelihood for Dependent Data , 1992 .

[38]  M. Tanner,et al.  [Practical Markov Chain Monte Carlo]: Comment: Monitoring Convergence of the Gibbs Sampler: Further Experience with the Gibbs Stopper , 1992 .

[39]  N. Sheehan,et al.  On the irreducibility of a Markov chain defined on a space of genotype configurations by a sampling scheme. , 1993, Biometrics.

[40]  Julian Besag,et al.  Towards Bayesian image analysis , 1993 .

[41]  J. Q. Smith,et al.  1. Bayesian Statistics 4 , 1993 .

[42]  Kung-Sik Chan Asymptotic behavior of the Gibbs sampler , 1993 .

[43]  A. Gelfand,et al.  Maximum-likelihood estimation for constrained- or missing-data models , 1993 .

[44]  Peter Green,et al.  Spatial statistics and Bayesian computation (with discussion) , 1993 .