Hierarchical sparse observation models and informative prior for Bayesian inference of spatially varying parameters

Abstract We develop a new computational approach for the speedy and accurate recovery of unknown spatially varying parameters of (stochastic) PDE models based on the sequentially arriving noisy observations of the evolving system variables. Two essential ingredients in our strategy for the identification of the target spatial fields are given by the Fourier diagonalization (FD) method and the hierarchical Bayesian model. We first apply the FD framework to perform an effectively parallelized computation in quantifying the propagating uncertainties of the dynamic variables in the Fourier space, and to facilitate a drastic saving of computational resources required for the Bayesian estimation of the associated parameters via sequential data assimilation. Yet our case study reveals that a very poor performance of the FD scheme occurs when the fast and slow variables coexist in the system evolution. The key observation is that, estimating the parameters in association with some slow variables, the FD-based Bayesian solver significantly underperforms compared to the remaining cases of faster variables. Due to this highly non-uniform discrepancy in the accuracy across distinct Fourier modes, the approximation of the parameter field cannot be so desired if the consequence is represented in the physical space. As one effort to circumvent this problem, we provide a systematic approach for a radical improvement of the naive FD technique. To do this, we make use of the hierarchical Bayesian model; it refers to the process of gradually enriching the knowledge of the unknown spatial fields from coarse to medium and to fine resolutions by using the Bayesian inference obtained at coarser levels to provide prior information for the estimation at finer levels. One major contribution of this paper is the development of a variant of the classical application of the multi-resolution approach through the close integration with the FD method, leading to the emergence of a new Bayesian paradigm for the data-driven parametric identification. Numerical experiments are performed to corroborate our demonstration concerning the efficacy and effectiveness of the proposed algorithm in obtaining a good degree of accuracy together with a significantly reduced computational cost.

[1]  A. Stordal,et al.  Bridging the ensemble Kalman filter and particle filters: the adaptive Gaussian mixture filter , 2011 .

[2]  Nicholas Zabaras,et al.  Hierarchical Bayesian models for inverse problems in heat conduction , 2005 .

[3]  P. Kitanidis Parameter Uncertainty in Estimation of Spatial Functions: Bayesian Analysis , 1986 .

[4]  A. Jazwinski Stochastic Processes and Filtering Theory , 1970 .

[5]  W. Strauss,et al.  Partial Differential Equations: An Introduction , 1992 .

[6]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[7]  M. Röckner,et al.  A Concise Course on Stochastic Partial Differential Equations , 2007 .

[8]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[9]  Eugenia Kalnay,et al.  Atmospheric Modeling, Data Assimilation and Predictability , 2002 .

[10]  Andrew J Majda,et al.  Explicit off-line criteria for stable accurate time filtering of strongly unstable spatially extended systems , 2007, Proceedings of the National Academy of Sciences.

[11]  Gardar Johannesson,et al.  Stochastic Engine Final Report: Applying Markov Chain Monte Carlo Methods with Importance Sampling to Large-Scale Data-Driven Simulation , 2004 .

[12]  Jun S. Liu,et al.  Mixture Kalman filters , 2000 .

[13]  Andrew J. Majda,et al.  Test Models for Filtering with Superparameterization , 2013, Multiscale Model. Simul..

[14]  Andrew J. Majda,et al.  Nonlinear Dynamics and Statistical Theories for Basic Geophysical Flows , 2006 .

[15]  B. Anderson,et al.  Optimal Filtering , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[16]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[17]  P. Bickel,et al.  Sharp failure rates for the bootstrap particle filter in high dimensions , 2008, 0805.3287.

[18]  J. Zabczyk,et al.  Stochastic Equations in Infinite Dimensions , 2008 .

[19]  I. Weir Fully Bayesian Reconstructions from Single-Photon Emission Computed Tomography Data , 1997 .

[20]  Andrew J. Majda,et al.  Filtering skill for turbulent signals for a suite of nonlinear and linear extended Kalman filters , 2012, J. Comput. Phys..

[21]  Mike West,et al.  Markov Random Field Models for High-Dimensional Parameters in Simulations of Fluid Flow in Porous Media , 2002, Technometrics.

[22]  D. M. Schmidt,et al.  Bayesian inference applied to the electromagnetic inverse problem , 1998, Human brain mapping.

[23]  P. Bickel,et al.  Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems , 2008, 0805.3034.

[24]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[25]  Ning Liu,et al.  Inverse Theory for Petroleum Reservoir Characterization and History Matching , 2008 .

[26]  Andrew J. Majda,et al.  Test models for improving filtering with model errors through stochastic parameter estimation , 2010, J. Comput. Phys..

[27]  Andrew J. Majda,et al.  Filtering Complex Turbulent Systems , 2012 .

[28]  Andrew J. Majda,et al.  Dynamic Stochastic Superresolution of sparsely observed turbulent systems , 2013, J. Comput. Phys..

[29]  Arthur Gelb,et al.  Applied Optimal Estimation , 1974 .

[30]  G. Evensen Data Assimilation: The Ensemble Kalman Filter , 2006 .

[31]  Ronald P. Barry,et al.  Blackbox Kriging: Spatial Prediction Without Specifying Variogram Models , 1996 .

[32]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[33]  Peijun Li,et al.  An inverse random source scattering problem in inhomogeneous media , 2011 .

[34]  Michael Goldstein,et al.  Bayesian Forecasting for Complex Systems Using Computer Simulators , 2001 .

[35]  Nicholas Zabaras,et al.  A Bayesian approach to multiscale inverse problems using the sequential Monte Carlo method , 2011 .

[36]  Phaedon-Stelios Koutsourelakis,et al.  A multi-resolution, non-parametric, Bayesian framework for identification of spatially-varying model parameters , 2008, J. Comput. Phys..

[37]  James O. Berger,et al.  A Bayesian analysis of the thermal challenge problem , 2008 .

[38]  A. Bennett Inverse Methods in Physical Oceanography , 1992 .

[39]  Andrew J. Majda,et al.  A statistically accurate modified quasilinear Gaussian closure for uncertainty quantification in turbulent dynamical systems , 2013 .

[40]  Andrew J. Majda,et al.  Filtering nonlinear dynamical systems with linear stochastic models , 2008 .

[41]  Andrew J. Majda,et al.  New Methods for Estimating Ocean Eddy Heat Transport Using Satellite Altimetry , 2012 .

[42]  H. Kushner Approximations to optimal nonlinear filters , 1967, IEEE Transactions on Automatic Control.

[43]  Andrew J. Majda,et al.  Mathematical test criteria for filtering complex systems: Plentiful observations , 2008, J. Comput. Phys..

[44]  P. Bickel,et al.  Obstacles to High-Dimensional Particle Filtering , 2008 .

[45]  Henning Omre,et al.  Uncertainty in Production Forecasts based on Well Observations, Seismic Data and Production History , 2001 .

[46]  Andrew J. Majda,et al.  Mathematical strategies for filtering turbulent dynamical systems , 2010 .

[47]  Andrew J. Majda,et al.  A NONLINEAR TEST MODEL FOR FILTERING SLOW-FAST SYSTEMS ∗ , 2008 .