Robust adaptive Metropolis algorithm with coerced acceptance rate

The adaptive Metropolis (AM) algorithm of Haario, Saksman and Tamminen (Bernoulli 7(2):223–242, 2001) uses the estimated covariance of the target distribution in the proposal distribution. This paper introduces a new robust adaptive Metropolis algorithm estimating the shape of the target distribution and simultaneously coercing the acceptance rate. The adaptation rule is computationally simple adding no extra cost compared with the AM algorithm. The adaptation strategy can be seen as a multidimensional extension of the previously proposed method adapting the scale of the proposal distribution in order to attain a given acceptance rate. The empirical results show promising behaviour of the new algorithm in an example with Student target distribution having no finite second moment, where the AM covariance estimate is unstable. In the examples with finite second moments, the performance of the new approach seems to be competitive with the AM algorithm combined with scale adaptation.

[1]  S. F. Jarner,et al.  Geometric ergodicity of Metropolis algorithms , 2000 .

[2]  G. Fort,et al.  Limit theorems for some adaptive MCMC algorithms with subgeometric kernels , 2008, 0807.2952.

[3]  G. Roberts,et al.  Convergence of Heavy‐tailed Monte Carlo Markov Chain Algorithms , 2007 .

[4]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[5]  V. Borkar.  Stochastic Approximation: A Dynamical Systems Viewpoint , 2008 .

[6]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[7]  C. Robert,et al.  Controlled MCMC for Optimal Sampling , 2001 .

[8]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[9]  Esa Nummelin,et al.  MC's for MCMC'ists , 2002 .

[10]  Eric Moulines,et al.  Stability of Stochastic Approximation under Verifiable Conditions , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[11]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[12]  E. Saksman,et al.  On the ergodicity of the adaptive Metropolis algorithm on unbounded domains , 2008, 0806.2933.

[13]  Matti Vihola,et al.  Can the Adaptive Metropolis Algorithm Collapse Without the Covariance Lower Bound , 2009, 0911.0522.

[14]  P. Rousseeuw,et al.  Wiley Series in Probability and Mathematical Statistics , 2005 .

[15]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[16]  Matti Vihola,et al.  Grapham: Graphical models with adaptive random walk Metropolis algorithms , 2008, Comput. Stat. Data Anal..

[17]  J. Rosenthal,et al.  On adaptive Markov chain Monte Carlo algorithms , 2005 .

[18]  A. Gelman,et al.  Weak convergence and optimal scaling of random walk Metropolis algorithms , 1997 .

[19]  Matti Vihola,et al.  On the stability and ergodicity of adaptive scaling Metropolis algorithms , 2009, 0903.4061.

[20]  Jack Dongarra,et al.  LINPACK Users' Guide , 1987 .

[21]  J. Rosenthal,et al.  General state space Markov chains and MCMC algorithms , 2004, math/0404033.

[22]  Convergence of stochastic approximation for Lyapunov stable dynamics : a proof from rst principles , 2004 .

[23]  J. Rosenthal,et al.  Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms , 2007, Journal of Applied Probability.

[24]  Gareth O. Roberts,et al.  Examples of Adaptive MCMC , 2009 .

[25]  David Hastie,et al.  Towards Automatic Reversible Jump Markov Chain Monte Carlo , 2005 .

[26]  Christophe Andrieu,et al.  A tutorial on adaptive MCMC , 2008, Stat. Comput..

[27]  J. Rosenthal,et al.  Optimal scaling for various Metropolis-Hastings algorithms , 2001 .

[28]  C. Andrieu,et al.  On the ergodicity properties of some adaptive MCMC algorithms , 2006, math/0610317.

[29]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[30]  Christian P. Robert,et al.  Monte Carlo Statistical Methods (Springer Texts in Statistics) , 2005 .

[31]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.