Anti-tempered layered adaptive importance sampling

Monte Carlo (MC) methods are widely used for Bayesian inference in signal processing, machine learning and statistics. In this work, we introduce an adaptive importance sampler which mixes together the benefits of the Importance Sampling (IS) and Markov Chain Monte Carlo (MCMC) approaches. Different parallel MCMC chains provide the location parameters of the proposal probability density functions (pdfs) used in an IS method. The MCMC algorithms consider a tempered version of the posterior distribution as invariant density. We also provide an exhaustive theoretical support explaining why, in the presented technique, even an anti-tempering strategy (reducing the scaling of the posterior) can be beneficial. Numerical results confirm the advantages of the proposed scheme.

[1]  A. Owen,et al.  Safe and Effective Importance Sampling , 2000 .

[2]  R. Douc,et al.  Convergence of Adaptive Sampling Schemes , 2007, 0708.0711.

[3]  R. Carroll,et al.  Advanced Markov Chain Monte Carlo Methods: Learning from Past Samples , 2010 .

[4]  Jukka Corander,et al.  An Adaptive Population Importance Sampler: Learning From Uncertainty , 2015, IEEE Transactions on Signal Processing.

[5]  Luca Martino,et al.  A novel rejection sampling scheme for posterior probability distributions , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[6]  Mónica F. Bugallo,et al.  Heretical Multiple Importance Sampling , 2016, IEEE Signal Processing Letters.

[7]  Luca Martino,et al.  Improving population Monte Carlo: Alternative weighting and resampling schemes , 2016, Signal Process..

[8]  Christophe Andrieu,et al.  A tutorial on adaptive MCMC , 2008, Stat. Comput..

[9]  Mónica F. Bugallo,et al.  Efficient Multiple Importance Sampling Estimators , 2015, IEEE Signal Processing Letters.

[10]  Frederik Beaujean,et al.  Initializing adaptive importance sampling with Markov chains , 2013, 1304.7808.

[11]  Joaquín Míguez,et al.  A population Monte Carlo scheme with transformed weights and its application to stochastic kinetic models , 2012, Stat. Comput..

[12]  Leonidas J. Guibas,et al.  Optimally combining sampling techniques for Monte Carlo rendering , 1995, SIGGRAPH.

[13]  Luca Martino,et al.  Fully adaptive Gaussian mixture Metropolis-Hastings algorithm , 2012, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Jean-Marie Cornuet,et al.  Adaptive Multiple Importance Sampling , 2009, 0907.1254.

[15]  Luca Martino,et al.  Almost rejectionless sampling from Nakagami-m distributions (m⩾1) , 2012 .

[16]  Jukka Corander,et al.  Layered adaptive importance sampling , 2015, Statistics and Computing.

[17]  Mónica F. Bugallo,et al.  Adaptive importance sampling in signal processing , 2015, Digit. Signal Process..

[18]  Jukka Corander,et al.  An adaptive population importance sampler , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  Luca Martino,et al.  Orthogonal MCMC algorithms , 2014, 2014 IEEE Workshop on Statistical Signal Processing (SSP).

[20]  David Luengo,et al.  Generalized Multiple Importance Sampling , 2015, Statistical Science.

[21]  Jean-Michel Marin,et al.  Convergence of Adaptive Sampling Schemes , 2004 .

[22]  J. Marin,et al.  Population Monte Carlo , 2004 .

[23]  N. Ushakov.  Selected Topics in Characteristic Functions , 1999 .

[24]  Bruno Tuffin,et al.  Markov chain importance sampling with applications to rare event probability estimation , 2011, Stat. Comput..

[25]  William J. Fitzgerald,et al.  Markov chain Monte Carlo methods with applications to signal processing , 2001, Signal Process..

[26]  T. Hesterberg,et al.  Weighted Average Importance Sampling and Defensive Mixture Distributions , 1995 .

[27]  F BugalloMónica,et al.  Improving population Monte Carlo , 2017 .

[28]  Luca Martino,et al.  Adaptive population importance samplers: A general perspective , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[29]  Jean-Michel Marin,et al.  Adaptive importance sampling in general mixture classes , 2007, Stat. Comput..