Bayesian model inversion using stochastic spectral embedding

In this paper we propose a new sampling-free approach to solve Bayesian inverse problems that extends the recently introduced spectral likelihood expansions (SLE) method. The latter solves the inverse problem by expanding the likelihood function onto a global polynomial basis orthogonal w.r.t. the prior distribution. This gives rise to analytical expressions for key statistics of the Bayesian posterior distribution, such as evidence, posterior moments and posterior marginals by simple post-processing of the expansion coefficients. It is well known that in most practically relevant scenarios, likelihood functions have close-to-compact support, which causes the global SLE approach to fail due to the high polynomial degree required for an accurate spectral representation. To solve this problem, we herein replace the global polynomial expansion from SLE with a recently proposed method for local spectral expansion refinement called stochastic spectral embedding (SSE). This surrogate-modeling method was developed for functions with high local complexity. To increase the efficiency of SSE, we enhance it with an adaptive sample enrichment scheme. We show that SSE works well for likelihood approximations and retains the relevant spectral properties of SLE, thus preserving analytical expressions of posterior statistics. To assess the performance of our approach, we include three case studies ranging from low to high dimensional model inversion problems that showcase the superiority of the SSE approach compared to SLE and present the approach as a promising alternative to existing inversion frameworks.

[1]  Bruno Sudret,et al.  Spectral likelihood expansions for Bayesian inference , 2015, J. Comput. Phys..

[2]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[3]  Stefano Marelli,et al.  Sequential Design of Experiment for Sparse Polynomial Chaos Expansions , 2017, SIAM/ASA J. Uncertain. Quantification.

[4]  Patrick R. Conrad,et al.  Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximations , 2014, 1402.1694.

[5]  Jonathan R Goodman,et al.  Ensemble samplers with affine invariance , 2010 .

[6]  J. Beck,et al.  Bayesian Updating of Structural Models and Reliability using Markov Chain Monte Carlo Simulation , 2002 .

[7]  Jian Guo Bayesian Inference—Data Evaluation and Decisions (2nd ed.) , 2019, Technometrics.

[8]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[9]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[10]  J. Beck,et al.  Updating Models and Their Uncertainties. I: Bayesian Statistical Framework , 1998 .

[11]  Radford M. Neal MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.

[12]  J. Ching,et al.  Transitional Markov Chain Monte Carlo Method for Bayesian Model Updating, Model Class Selection, and Model Averaging , 2007 .

[13]  D. Xiu Numerical Methods for Stochastic Computations: A Spectral Method Approach , 2010 .

[14]  Hanns Ludwig Harney Bayesian Inference: Data Evaluation and Decisions , 2016 .

[15]  L. Tierney,et al.  Approximate marginal densities of nonlinear functions , 1989 .

[16]  Tiangang Cui,et al.  Data‐driven model reduction for the Bayesian solution of inverse problems , 2014, 1403.4290.

[17]  Jean-Michel Marin,et al.  Approximate Bayesian computational methods , 2011, Statistics and Computing.

[18]  Stefano Marelli,et al.  Sparse Polynomial Chaos Expansions: Literature Survey and Benchmark , 2021, SIAM/ASA J. Uncertain. Quantification.

[19]  Andrew Gelman,et al.  Handbook of Markov Chain Monte Carlo , 2011 .

[20]  Michael Peters,et al.  Higher-Order Quasi-Monte Carlo for Bayesian Shape Inversion , 2018, SIAM/ASA J. Uncertain. Quantification.

[21]  Jianhua Lin,et al.  Divergence measures based on the Shannon entropy , 1991, IEEE Trans. Inf. Theory.

[22]  B. Sudret,et al.  An adaptive algorithm to build up sparse polynomial chaos expansions for stochastic finite element analysis , 2010 .

[23]  Matthew Parno,et al.  Transport maps for accelerated Bayesian computation , 2015 .

[24]  Liang Yan,et al.  Adaptive multi-fidelity polynomial chaos approach to Bayesian inference in inverse problems , 2018, J. Comput. Phys..

[25]  Khachik Sargsyan,et al.  Enhancing ℓ1-minimization estimates of polynomial chaos expansions using basis selection , 2014, J. Comput. Phys..

[26]  W. Gautschi Orthogonal Polynomials: Computation and Approximation , 2004 .

[27]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[28]  H. Jeffreys An invariant form for the prior probability in estimation problems , 1946, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[29]  B. Sudret,et al.  Bayesian calibration and sensitivity analysis of heat transfer models for fire insulation panels , 2019, Engineering Structures.

[30]  Stefano Marelli,et al.  Data-driven polynomial chaos expansion for machine learning regression , 2018, J. Comput. Phys..

[31]  Aaron Smith,et al.  Parallel Local Approximation MCMC for Expensive Models , 2016, SIAM/ASA J. Uncertain. Quantification.

[32]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[33]  Yanan Fan,et al.  Handbook of Approximate Bayesian Computation , 2018 .

[34]  L. Tierney,et al.  Accurate Approximations for Posterior Moments and Marginal Densities , 1986 .

[35]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[36]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[37]  Youssef M. Marzouk,et al.  Bayesian inference with optimal maps , 2011, J. Comput. Phys..

[38]  L. Tierney,et al.  Fully Exponential Laplace Approximations to Expectations and Variances of Nonpositive Functions , 1989 .

[39]  Josef Dick,et al.  Multilevel higher-order quasi-Monte Carlo Bayesian estimation , 2016, 1611.08324.

[40]  Stefano Marelli,et al.  UQLab: a framework for uncertainty quantification in MATLAB , 2014 .

[41]  Jinglai Li,et al.  Adaptive Construction of Surrogates for the Bayesian Solution of Inverse Problems , 2013, SIAM J. Sci. Comput..

[42]  Y. Marzouk,et al.  A stochastic collocation approach to Bayesian inference in inverse problems , 2009 .

[43]  K. J. Keen Texts in Statistical Science , 2018 .

[44]  E. Jaynes Probability theory : the logic of science , 2003 .

[45]  Dongbin Xiu,et al.  Nonadaptive Quasi-Optimal Points Selection for Least Squares Linear Regression , 2016, SIAM J. Sci. Comput..

[46]  Yoshua Bengio,et al.  Model Selection for Small Sample Regression , 2002, Machine Learning.

[47]  Habib N. Najm,et al.  Stochastic spectral methods for efficient Bayesian solution of inverse problems , 2005, J. Comput. Phys..

[48]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[49]  R. Ghanem,et al.  Stochastic Finite Elements: A Spectral Approach , 1990 .

[50]  Matthew P. Wand,et al.  Kernel Smoothing , 1995 .

[51]  D. Lucor,et al.  Adaptive Bayesian Inference for Discontinuous Inverse Problems, Application to Hyperbolic Conservation Laws , 2014 .

[52]  S. Marelli,et al.  STOCHASTIC SPECTRAL EMBEDDING , 2020, International Journal for Uncertainty Quantification.

[53]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[54]  M. Rosenblatt Remarks on a Multivariate Transformation , 1952 .

[55]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[56]  Bruno Sudret,et al.  Adaptive sparse polynomial chaos expansion based on least angle regression , 2011, J. Comput. Phys..