General state space Markov chains and MCMC algorithms

This paper surveys various results about Markov chains on gen- eral (non-countable) state spaces. It begins with an introduction to Markov chain Monte Carlo (MCMC) algorithms, which provide the motivation and context for the theory which follows. Then, sucient conditions for geomet- ric and uniform ergodicity are presented, along with quantitative bounds on the rate of convergence to stationarity. Many of these results are proved using direct coupling constructions based on minorisation and drift con- ditions. Necessary and sucient conditions for Central Limit Theorems (CLTs) are also presented, in some cases proved via the Poisson Equa- tion or direct regeneration constructions. Finally, optimal scaling and weak convergence results for Metropolis-Hastings algorithms are discussed. None of the results presented is new, though many of the proofs are. We also describe some Open Problems.

[1]  A. Sapozhnikov.  Subgeometric rates of convergence of f-ergodic Markov chains , 2006 .

[2]  J. Rosenthal,et al.  On adaptive Markov chain Monte Carlo algorithms , 2005 .

[3]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[4]  Olle Häggström,et al.  On the central limit theorem for geometrically ergodic Markov chains , 2005 .

[5]  J. Rosenthal,et al.  Scaling limits for the transient phase of local Metropolis–Hastings algorithms , 2005 .

[6]  P. Baxendale.  Renewal theory and computable convergence rates for geometrically ergodic Markov chains , 2005, math/0503515.

[7]  R. Douc,et al.  Quantitative bounds on convergence of time-inhomogeneous Markov chains , 2004, math/0503532.

[8]  Radford M. Neal.  Improving Asymptotic Variance of MCMC Estimators: Non-reversible Chains are Better , 2004, math/0407281.

[9]  Galin L. Jones,et al.  Sufficient burn-in for Gibbs samplers for a hierarchical random effects model , 2004, math/0406454.

[10]  J. Rosenthal.  Geometric Convergence Rates for Time-Sampled Markov Chains , 2003 .

[11]  É. Moulines,et al.  Polynomial ergodicity of Markov transition kernels , 2003 .

[12]  Central Limit Theorem for Markov Processes , 2003 .

[13]  Computable bounds for V-geometric ergodicity of Markov transition kernels , 2003 .

[14]  Galin L. Jones,et al.  On the applicability of regenerative simulation in Markov chain Monte Carlo , 2002 .

[15]  J. Rosenthal.  QUANTITATIVE CONVERGENCE RATES OF MARKOV CHAINS: A SIMPLE ACCOUNT , 2002 .

[16]  G. Roberts,et al.  Polynomial convergence rates of Markov chains. , 2002 .

[17]  S. Rosenthal,et al.  A review of asymptotic convergence for general state space Markov chains , 2002 .

[18]  S. Rosenthal,et al.  Asymptotic Variance and Convergence Rates of Nearly-Periodic MCMC Algorithms , 2002 .

[19]  J. Rosenthal,et al.  Optimal scaling for various Metropolis-Hastings algorithms , 2001 .

[20]  Gareth O. Roberts,et al.  Markov Chains and De‐initializing Processes , 2001 .

[21]  Gareth O. Roberts,et al.  SMALL AND PSEUDO-SMALL SETS FOR MARKOV CHAINS , 2001 .

[22]  R. Lund,et al.  Geometric renewal convergence rates from hazard rates , 2001, Journal of Applied Probability.

[23]  Gareth O. Roberts,et al.  Corrigendum to : Bounds on regeneration times and convergence rates for Markov chains , 2001 .

[24]  Galin L. Jones,et al.  Honest Exploration of Intractable Probability Distributions via Markov Chain Monte Carlo , 2001 .

[25]  J. Rosenthal,et al.  Markov Chains and De-initialising Processes by , 2001 .

[26]  Gareth O. Roberts,et al.  From metropolis to diffusions: Gibbs states and optimal scaling , 2000 .

[27]  J. Rosenthal,et al.  Extension of Fill's perfect rejection sampling algorithm to general chains , 2000, Random Struct. Algorithms.

[28]  W. Kendall,et al.  Perfect simulation using dominating processes on ordered spaces, with application to locally stable point processes , 2000, Advances in Applied Probability.

[29]  J. Rosenthal.  A First Look at Rigorous Probability Theory , 2000 .

[30]  H. Thorisson.  Coupling, stationarity, and regeneration , 2000 .

[31]  G. Roberts.  A note on acceptance rate criteria for CLTS for Metropolis–Hastings algorithms , 1999, Journal of Applied Probability.

[32]  J. Rosenthal,et al.  Possible biases induced by MCMC convergence diagnostics , 1999 .

[33]  J. Møller,et al.  Log Gaussian Cox Processes , 1998 .

[34]  Mary Kathryn Cowles,et al.  A simulation approach to convergence rates for Markov chain Monte Carlo algorithms , 1998, Stat. Comput..

[35]  Gareth O. Roberts,et al.  Markov‐chain monte carlo: Some practical implications of theoretical results , 1998 .

[36]  G. Roberts.  Optimal metropolis algorithms for product measures on the vertices of a hypercube , 1998 .

[37]  J. Rosenthal,et al.  Optimal scaling of discrete approximations to Langevin diffusions , 1998 .

[38]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[39]  J. Rosenthal,et al.  Geometric Ergodicity and Hybrid Markov Chains , 1997 .

[40]  A. Gelman,et al.  Weak convergence and optimal scaling of random walk Metropolis algorithms , 1997 .

[41]  J. Rosenthal,et al.  Shift-coupling and convergence rates of ergodic averages , 1997 .

[42]  David Bruce Wilson,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996, Random Struct. Algorithms.

[43]  R. Tweedie,et al.  Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms , 1996 .

[44]  S. Meyn,et al.  Computable exponential convergence rates for stochastically ordered Markov processes , 1996 .

[45]  R. Tweedie,et al.  Rates of convergence of the Hastings and Metropolis algorithms , 1996 .

[46]  K. Athreya,et al.  ON THE CONVERGENCE OF THE MARKOV CHAIN SIMULATION METHOD , 1996 .

[47]  Jeffrey S. Rosenthal,et al.  Convergence Rates for Markov Chains , 1995, SIAM Rev..

[48]  J. Rosenthal.  RATES OF CONVERGENCE FOR GIBBS SAMPLING FOR VARIANCE COMPONENT MODELS , 1995 .

[49]  Bin Yu,et al.  Regeneration in Markov chain samplers , 1995 .

[50]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo conver-gence diagnostics: a comparative review , 1996 .

[51]  J. Rosenthal.  Minorization Conditions and Convergence Rates for Markov Chain Monte Carlo , 1995 .

[52]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[53]  S. Meyn,et al.  Computable Bounds for Geometric Convergence Rates of Markov Chains , 1994 .

[54]  J. Rosenthal.  Rates of Convergence for Data Augmentation on Finite Sample Spaces , 1993 .

[55]  P. Matthews.  A slowly mixing Markov chain with implications for Gibbs sampling , 1993 .

[56]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[57]  David Aldous,et al.  Inequalities for rare events in time-reversible Markov chains II , 1993 .

[58]  Alistair Sinclair,et al.  Improved Bounds for Mixing Rates of Markov Chains and Multicommodity Flow , 1992, Combinatorics, Probability and Computing.

[59]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[60]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[61]  T. Lindvall.  Lectures on the Coupling Method , 1992 .

[62]  Alistair Sinclair,et al.  Improved Bounds for Mixing Rates of Marked Chains and Multicommodity Flow , 1992, LATIN.

[63]  R. Durrett.  Probability: Theory and Examples , 1993 .

[64]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[65]  Sلأren Asmussen,et al.  Applied Probability and Queues , 1989 .

[66]  P. Diaconis.  Group representations in probability and statistics , 1988 .

[67]  Adrian F. M. Smith,et al.  Bayesian computation via the gibbs sampler and related markov chain monte carlo methods (with discus , 1993 .

[68]  Upendra Dave,et al.  Applied Probability and Queues , 1987 .

[69]  P. Diaconis,et al.  Strong uniform times and finite random walks , 1987 .

[70]  S. Varadhan,et al.  Central limit theorem for additive functionals of reversible Markov processes and applications to simple exclusions , 1986 .

[71]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[72]  E. Nummelin.  General irreducible Markov chains and non-negative operators: Embedded renewal processes , 1984 .

[73]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.

[74]  K. Athreya,et al.  A New Approach to the Limit Theory of Recurrent Markov Chains , 1978 .

[75]  E. Nummelin.  Uniform and ratio limit theorems for Markov renewal and semi-regenerative processes on a general state space , 1978 .

[76]  J. Pitman.  On coupling of Markov chains , 1976 .

[77]  C. Stein.  A bound for the error in the normal approximation to the distribution of a sum of dependent random variables , 1972 .

[78]  I. Ibragimov,et al.  Independent and stationary sequences of random variables , 1971 .

[79]  S. Orey.  Lecture Notes on Limit Theorems for Markov Chain Transition Probabilities , 1971 .

[80]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[81]  B. Jamison,et al.  Contributions to Doeblin's theory of Markov processes , 1967 .

[82]  I. Ibragimov.  A Central Limit Theorem for a Class of Dependent Random Variables , 1963 .

[83]  P. Billingsley,et al.  The Lindeberg-Lévy theorem for martingales , 1961 .

[84]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[85]  J. Doob.  Stochastic processes , 1953 .