Multilevel Sequential Monte Carlo Samplers for Normalizing Constants

This article considers the Sequential Monte Carlo (SMC) approximation of ratios of normalizing constants associated to posterior distributions which in principle rely on continuum models. Therefore, the Monte Carlo estimation error and the discrete approximation error must be balanced. A multilevel strategy is utilized to substantially reduce the cost to obtain a given error level in the approximation as compared to standard estimators. Two estimators are considered and relative variance bounds are given. The theoretical results are numerically illustrated for two Bayesian inverse problems arising from elliptic Partial Differential Equations (PDEs). The examples involve the inversion of observations of the solution of (i) a one-dimensional Poisson equation to infer the diffusion coefficient, and (ii) a two-dimensional Poisson equation to infer the external forcing.

[1]  B. Øksendal.  Stochastic Differential Equations , 1985 .

[2]  P. Bickel,et al.  Sharp failure rates for the bootstrap particle filter in high dimensions , 2008, 0805.3287.

[3]  Adrian E. Raftery,et al.  Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors , 1999 .

[4]  H. Bungartz,et al.  Sparse grids , 2004, Acta Numerica.

[5]  A. Beskos,et al.  On the stability of sequential Monte Carlo methods in high dimensions , 2011, 1103.3965.

[6]  Ajay Jasra,et al.  Forward and Inverse Uncertainty Quantification using Multilevel Monte Carlo Algorithms for an Elliptic Nonlocal Equation , 2016, 1603.06381.

[7]  Kody J. H. Law,et al.  Multilevel ensemble Kalman filtering , 2015, SIAM J. Numer. Anal..

[8]  Wasserman,et al.  Bayesian Model Selection and Model Averaging. , 2000, Journal of mathematical psychology.

[9]  Stefan Heinrich,et al.  Multilevel Monte Carlo Methods , 2001, LSSC.

[10]  K. Kamatani,et al.  Multilevel particle filter , 2015 .

[11]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.

[12]  Kody J. H. Law,et al.  Multilevel sequential Monte Carlo: Mean square error bounds under verifiable conditions , 2017 .

[13]  P. Moral,et al.  A nonasymptotic theorem for unnormalized Feynman-Kac particle models , 2011 .

[14]  Richard Bellman,et al.  Adaptive Control Processes - A Guided Tour (Reprint from 1961) , 2015, Princeton Legacy Library.

[15]  G. Roberts,et al.  MCMC Methods for Functions: ModifyingOld Algorithms to Make Them Faster , 2012, 1202.0709.

[16]  Peter W. Glynn,et al.  Unbiased Estimation with Square Root Convergence for SDE Models , 2015, Oper. Res..

[17]  J. E. Walstrom,et al.  Evaluating Uncertainty in Engineering Calculations , 1967 .

[18]  Andrew M. Stuart,et al.  Complexity analysis of accelerated MCMC methods for Bayesian inversion , 2012, 1207.2411.

[19]  Adrian E. Raftery,et al.  Bayesian Model Averaging: A Tutorial , 2016 .

[20]  Hans-Joachim Bungartz,et al.  Acta Numerica 2004: Sparse grids , 2004 .

[21]  Yan Zhou,et al.  Toward Automatic Model Comparison: An Adaptive Sequential Monte Carlo Approach , 2016 .

[22]  T. J. Dodwell,et al.  A Hierarchical Multilevel Markov Chain Monte Carlo Algorithm with Applications to Uncertainty Quantification in Subsurface Flow , 2013, SIAM/ASA J. Uncertain. Quantification.

[23]  P. Moral.  Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[24]  C. Andrieu,et al.  The pseudo-marginal approach for efficient Monte Carlo computations , 2009, 0903.5480.

[25]  Stig Larsson,et al.  Introduction to stochastic partial differential equations , 2008 .

[26]  Omar M. Knio,et al.  Introduction: Uncertainty Quantification and Propagation , 2010 .

[27]  Michael B. Giles,et al.  Multilevel Monte Carlo Path Simulation , 2008, Oper. Res..

[28]  Claude Jeffrey Gittelson,et al.  Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs* , 2011, Acta Numerica.

[29]  A. Beskos,et al.  Multilevel sequential Monte Carlo samplers , 2015, 1503.07259.

[30]  Philippe G. Ciarlet,et al.  The finite element method for elliptic problems , 2002, Classics in applied mathematics.

[31]  Nikolaus Schweizer.  Non-asymptotic Error Bounds for Sequential MCMC and Stability of Feynman-Kac Propagators , 2012, 1204.2382.

[32]  A. M. Johansen,et al.  Towards Automatic Model Comparison: An Adaptive Sequential Monte Carlo Approach , 2013, 1303.3123.