Parameter estimation by implicit sampling

Author(s): Morzfeld, M; Tu, X; Wilkening, J; Chorin, AJ | Abstract: © 2015 Mathematical Sciences Publishers. Implicit sampling is a weighted sampling method that is used in data assimilation to sequentially update state estimates of a stochastic model based on noisy and incomplete data. Here we apply implicit sampling to sample the posterior probability density of parameter estimation problems. The posterior probability combines prior information about the parameter with information from a numerical model, e.g., a partial differential equation (PDE), and noisy data. The result of our computations are parameters that lead to simulations that are compatible with the data. We demonstrate the usefulness of our implicit sampling algorithm with an example from subsurface flow. For an efficient implementation, we make use of multiple grids, BFGS optimization coupled to adjoint equations, and Karhunen-Loeve expansions for dimensional reduction. Several difficulties of Markov chain Monte Carlo methods, e.g., estimation of burn-in times or correlations among the samples, are avoided because the implicit samples are independent.

[1]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[2]  James Martin,et al.  A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion , 2012, SIAM J. Sci. Comput..

[3]  A. Chorin,et al.  Implicit particle filtering for models with partial noise, and an application to geomagnetic data assimilation , 2011, 1109.3664.

[4]  James Martin,et al.  A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems Part I: The Linearized Case, with Application to Global Seismic Inversion , 2013, SIAM J. Sci. Comput..

[5]  Brad Weir,et al.  Implicit Estimation of Ecological Model Parameters , 2013, Bulletin of mathematical biology.

[6]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[7]  A. Chorin,et al.  Implicit Particle Methods and Their Connection with Variational Data Assimilation , 2012, 1205.1830.

[8]  Matthias Morzfeld,et al.  Conditions for successful data assimilation , 2013, 1303.2714.

[9]  Ning Liu,et al.  Inverse Theory for Petroleum Reservoir Characterization and History Matching , 2008 .

[10]  D. Braess.  Finite Elements: Theory, Fast Solvers, and Applications in Solid Mechanics , 1995 .

[11]  Marco A. Iglesias,et al.  Evaluation of Gaussian approximations for data assimilation in reservoir models , 2012, Computational Geosciences.

[12]  Stefan Ulbrich,et al.  Optimization with PDE Constraints , 2008, Mathematical modelling.

[13]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[14]  Matthias Morzfeld,et al.  Implicit particle filters for data assimilation , 2010, 1005.4002.

[15]  R. Ghanem,et al.  Stochastic Finite Elements: A Spectral Approach , 1990 .

[16]  Omar M. Knio,et al.  Spectral Methods for Uncertainty Quantification , 2010 .

[17]  A. Chorin,et al.  Stochastic Tools in Mathematics and Science , 2005 .

[18]  Eric Vanden-Eijnden,et al.  DATA ASSIMILATION IN THE LOW NOISE, ACCURATE OBSERVATION REGIME WITH APPLICATION TO THE KUROSHIO CURRENT , 2012 .

[19]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[20]  Kristine L. Bell,et al.  A Tutorial on Particle Filters for Online Nonlinear/NonGaussian Bayesian Tracking , 2007 .

[21]  Matthias Morzfeld,et al.  Small-noise analysis and symmetrization of implicit Monte Carlo samplers , 2014 .

[22]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[23]  Y. Marzouk,et al.  A stochastic collocation approach to Bayesian inference in inverse problems , 2009 .

[24]  Rémi Bardenet,et al.  Monte Carlo Methods , 2013, Encyclopedia of Social Network Analysis and Mining. 2nd Ed..

[25]  A. Stuart,et al.  Ensemble Kalman methods for inverse problems , 2012, 1209.2736.

[26]  Jun S. Liu,et al.  Blind Deconvolution via Sequential Imputations , 1995 .

[27]  P. Bickel,et al.  Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems , 2008, 0805.3034.

[28]  P. Bickel,et al.  Sharp failure rates for the bootstrap particle filter in high dimensions , 2008, 0805.3287.

[29]  Jonathan R Goodman,et al.  Ensemble samplers with affine invariance , 2010 .

[30]  E. Loh.  Multigrid Monte Carlo methods , 1988 .

[31]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[32]  Leonard F. Konikow,et al.  Modeling Groundwater Flow and Pollution , 1988 .

[33]  E. Vanden-Eijnden,et al.  Data Assimilation in the Low Noise Regime with Application to the Kuroshio , 2012, 1202.4952.

[34]  Dean S. Oliver.  Minimization for conditional simulation: Relationship to optimal transport , 2014, J. Comput. Phys..

[35]  P. Bickel,et al.  Curse-of-dimensionality revisited : Collapse of importance sampling in very large scale systems , 2005 .

[36]  O. L. Maître,et al.  Spectral Methods for Uncertainty Quantification: With Applications to Computational Fluid Dynamics , 2010 .

[37]  A. Chorin,et al.  Implicit Sampling, with Application to Data Assimilation , 2013, Chinese Annals of Mathematics, Series B.

[38]  A. Chorin,et al.  Implicit sampling for particle filters , 2009, Proceedings of the National Academy of Sciences.

[39]  Yalchin Efendiev,et al.  Preconditioning Markov Chain Monte Carlo Simulations Using Coarse-Scale Models , 2006, SIAM J. Sci. Comput..

[40]  P. Bickel,et al.  Obstacles to High-Dimensional Particle Filtering , 2008 .

[41]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[42]  S OliverDean.  Minimization for conditional simulation , 2014 .

[43]  CLARK R. DOHRMANN,et al.  A Preconditioner for Substructuring Based on Constrained Energy Minimization , 2003, SIAM J. Sci. Comput..

[44]  James Martin,et al.  A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems, Part II: Stochastic Newton MCMC with Application to Ice Sheet Flow Inverse Problems , 2013, SIAM J. Sci. Comput..

[45]  D. Oliver,et al.  Recent progress on reservoir history matching: a review , 2011 .

[46]  Matthias Morzfeld,et al.  A random map implementation of implicit filters , 2011, J. Comput. Phys..

[47]  Habib N. Najm,et al.  Stochastic spectral methods for efficient Bayesian solution of inverse problems , 2005, J. Comput. Phys..

[48]  Andrew M. Stuart,et al.  Uncertainty Quantification and Weak Approximation of an Elliptic Inverse Problem , 2011, SIAM J. Numer. Anal..

[49]  R. P. Fedorenko.  A relaxation method for solving elliptic difference equations , 1962 .

[50]  Habib N. Najm,et al.  Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems , 2008, J. Comput. Phys..

[51]  Youssef M. Marzouk,et al.  Bayesian inference with optimal maps , 2011, J. Comput. Phys..

[52]  Goodman,et al.  Multigrid Monte Carlo method. Conceptual foundations. , 1989, Physical review. D, Particles and fields.