Limitations of polynomial chaos expansions in the Bayesian solution of inverse problems

Polynomial chaos expansions are used to reduce the computational cost in the Bayesian solutions of inverse problems by creating a surrogate posterior that can be evaluated inexpensively. We show, by analysis and example, that when the data contain significant information beyond what is assumed in the prior, the surrogate posterior can be very different from the posterior, and the resulting estimates become inaccurate. One can improve the accuracy by adaptively increasing the order of the polynomial chaos, but the cost may increase too fast for this to be cost effective compared to Monte Carlo sampling without a surrogate posterior.

[1]  Francis Comets,et al.  Large Deviations and Applications , 2011, International Encyclopedia of Statistical Science.

[2]  R. Ghanem,et al.  Stochastic Finite Elements: A Spectral Approach , 1990 .

[3]  Andrew M. Stuart,et al.  Uncertainty Quantification and Weak Approximation of an Elliptic Inverse Problem , 2011, SIAM J. Numer. Anal..

[4]  Andrew J. Majda,et al.  Fundamental limitations of polynomial chaos for uncertainty quantification in systems with intermittent instabilities , 2013 .

[5]  W. Schoutens.  Stochastic processes and orthogonal polynomials , 2000 .

[6]  A. Chorin,et al.  Implicit sampling for particle filters , 2009, Proceedings of the National Academy of Sciences.

[7]  Habib N. Najm,et al.  Uncertainty Quantification and Polynomial Chaos Techniques in Computational Fluid Dynamics , 2009 .

[8]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[9]  Panos G. Georgopoulos,et al.  Uncertainty reduction and characterization for complex environmental fate and transport models: An empirical Bayesian framework incorporating the stochastic response surface method , 2003 .

[10]  Thomas Y. Hou,et al.  Wiener Chaos expansions and numerical solutions of randomly forced equations of fluid mechanics , 2006, J. Comput. Phys..

[11]  E. Vanden-Eijnden,et al.  Rare Event Simulation of Small Noise Diffusions , 2012 .

[12]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[13]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[14]  Guo Ben-yu,et al.  Hermite spectral and pseudospectral methods for nonlinear partial differential equation in multiple dimensions , 2003 .

[15]  N. Zabaras,et al.  Solution of inverse problems with limited forward solver evaluations: a Bayesian perspective , 2013 .

[16]  A. Chorin,et al.  Stochastic Tools in Mathematics and Science , 2005 .

[17]  A. Chorin,et al.  Implicit Particle Methods and Their Connection with Variational Data Assimilation , 2012, 1205.1830.

[18]  Matthias Morzfeld,et al.  Parameter estimation by implicit sampling , 2013, 1308.4640.

[19]  Khachik Sargsyan,et al.  Uncertainty Quantification in MD Simulations. Part II: Bayesian Inference of Force-Field Parameters , 2012, Multiscale Model. Simul..

[20]  Faming Liang,et al.  Statistical and Computational Inverse Problems , 2006, Technometrics.

[21]  D. Xiu.  Numerical Methods for Stochastic Computations: A Spectral Method Approach , 2010 .

[22]  A. Chorin.  Gaussian fields and random flow , 1974, Journal of Fluid Mechanics.

[23]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[24]  O. L. Maître,et al.  Spectral Methods for Uncertainty Quantification: With Applications to Computational Fluid Dynamics , 2010 .

[25]  Matthias Morzfeld,et al.  A random map implementation of implicit filters , 2011, J. Comput. Phys..

[26]  Habib N. Najm,et al.  Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems , 2008, J. Comput. Phys..

[27]  Y. Marzouk,et al.  A stochastic collocation approach to Bayesian inference in inverse problems , 2009 .

[28]  D. Braess.  Finite Elements: Theory, Fast Solvers, and Applications in Solid Mechanics , 1995 .

[29]  Habib N. Najm,et al.  Stochastic spectral methods for efficient Bayesian solution of inverse problems , 2005, J. Comput. Phys..

[30]  E. Vanden-Eijnden,et al.  Data Assimilation in the Low Noise Regime with Application to the Kuroshio , 2012, 1202.4952.

[31]  Matthias Morzfeld,et al.  Implicit particle filters for data assimilation , 2010, 1005.4002.

[32]  G. H. Canavan,et al.  Relationship between a Wiener–Hermite expansion and an energy cascade , 1970, Journal of Fluid Mechanics.