On the ergodicity properties of some adaptive MCMC algorithms

In this paper we study the ergodicity properties of some adaptive Markov chain Monte Carlo algorithms (MCMC) that have been recently proposed in the literature. We prove that under a set of verifiable conditions, ergodic averages calculated from the output of a so-called adaptive MCMC sampler converge to the required value and can even, under more stringent assumptions, satisfy a central limit theorem. We prove that the conditions required are satisfied for the independent Metropolis–Hastings algorithm and the random walk Metropolis algorithm with symmetric increments. Finally, we propose an application of these results to the case where the proposal distribution of the Metropolis–Hastings update is a mixture of distributions from a curved exponential family.

[1]  J. Doob.  Stochastic processes , 1953 .

[2]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[3]  Z. Birnbaum,et al.  SOME MULTIVARIATE CHEBYSHEV INEQUALITIES WITH EXTENSIONS TO CONTINUOUS PARAMETER PROCESSES , 1961 .

[4]  Michel Loève,et al.  Probability Theory I , 1977 .

[5]  A. Dvoretzky,et al.  Asymptotic normality for sums of dependent random variables , 1972 .

[6]  D. McLeish.  A Maximal Inequality and Dependent Strong Laws , 1975 .

[7]  P. Hall.  Martingale Invariance Principles , 1977 .

[8]  P. Hall,et al.  Rates of Convergence in the Martingale Central Limit Theorem , 1981 .

[9]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[10]  P. Hall,et al.  Martingale Limit Theory and its Application. , 1984 .

[11]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[12]  H. Chen,et al.  STOCHASTIC APPROXIMATION PROCEDURES WITH RANDOMLY VARYING TRUNCATIONS , 1986 .

[13]  Han-Fu Chen,et al.  Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds , 1987 .

[14]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[15]  E. Nummelin.  On the Poisson equation in the potential theory of a single kernel. , 1991 .

[16]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[17]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[18]  S. Meyn,et al.  Computable Bounds for Geometric Convergence Rates of Markov Chains , 1994 .

[19]  R. Tweedie,et al.  Rates of convergence of the Hastings and Metropolis algorithms , 1996 .

[20]  R. Tweedie,et al.  Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms , 1996 .

[21]  Sean P. Meyn,et al.  A Liapounov bound for solutions of the Poisson equation , 1996 .

[22]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[23]  Xiao-Li Meng,et al.  The EM Algorithm—an Old Folk‐song Sung to a Fast New Tune , 1997 .

[24]  Lars Holden,et al.  Adaptive Chains , 1998 .

[25]  G. Roberts,et al.  Adaptive Markov Chain Monte Carlo through Regeneration , 1998 .

[26]  S. F. Jarner,et al.  Geometric ergodicity of Metropolis algorithms , 2000 .

[27]  John D. Bartusek.  Stochastic Approximation and Optimization for Markov Chains , 2000 .

[28]  C. Robert,et al.  Controlled MCMC for Optimal Sampling , 2001 .

[29]  Gareth O. Roberts,et al.  Corrigendum to : Bounds on regeneration times and convergence rates for Markov chains , 2001 .

[30]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[31]  R. Douc,et al.  Quantitative bounds for geometric convergence rates of Markov Chains , 2002 .

[32]  J. Gåsemyr.  On an adaptive version of the Metropolis-Hastings algorithm with independent proposal distribution , 2003 .

[33]  Peter Arcidiacono,et al.  Finite Mixture Distributions, Sequential Likelihood and the EM Algorithm , 2003 .

[34]  Simon J. Godsill,et al.  Dicussion on the meeting on ‘Statistical approaches to inverse problems’ , 2004 .

[35]  Eric Moulines,et al.  Stability of Stochastic Approximation under Verifiable Conditions , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[36]  J. Rosenthal,et al.  On adaptive Markov chain Monte Carlo algorithms , 2005 .

[37]  P. Baxendale.  Renewal theory and computable convergence rates for geometrically ergodic Markov chains , 2005, math/0503515.

[38]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[39]  B. Craven.  Control and optimization , 2019, Mathematical Modelling of the Human Cardiovascular System.