Importance sampling and delayed acceptance via a Peskun type ordering

We consider importance sampling (IS), pseudomarginal (PM), and delayed acceptance (DA) approaches to reversible Markov chain Monte Carlo, and compare the asymptotic variances. Despite their similarity in terms of using an approximation or unbiased estimators, not much has been said about the relative efficiency of IS and PM/DA. Simple examples demonstrate that the answer is setting specific. We show that the IS asymptotic variance is strictly less than a constant times the PM/DA asymptotic variance, where the constant is twice the essential supremum of the importance weight, and that the inequality becomes an equality as the weight approaches unity in the uniform norm. A version of the inequality also holds for the case of unbounded weight estimators, as long as the estimators are square integrable. The result, together with robustness and computational cost considerations in the context of parallel computing, lends weight to the suggestion of using IS over that of PM/DA when reasonable approximations are available. Our result is based on a Peskun type ordering for importance sampling, which, assuming a familiar Dirichlet form inequality, bounds the asymptotic variance of an IS correction with that of any reversible Markov chain admitting the target distribution as a marginal of its stationary measure. Our results apply to chains using unbiased estimators, which accommodates compound sampling and pseudomarginal settings.