A McKean optimal transportation perspective on Feynman-Kac formulae with application to data assimilation

Data assimilation is the task of combining mathematical models with observational data. From a mathematical perspective data assimilation leads to Bayesian inference problems which can be formulated in terms of Feynman-Kac formulae. In this paper we focus on the sequential nature of many data assimilation problems and their numerical implementation in form of Monte Carlo methods. We demonstrate how sequential data assimilation can be interpreted as time-dependent Markov processes, which is often referred to as the McKean approach to Feynman-Kac formulae. It is shown that the McKean approach has very natural links to coupling of random variables and optimal transportation. This link allows one to propose novel sequential Monte Carlo methods/particle filters. In combination with localization these novel algorithms have the potential of beating the curse of dimensionality, which has prevented particle filters from being applied to spatially extended systems.

[1]  D. Crisan,et al.  Approximate McKean–Vlasov representations for a class of SPDEs , 2005, math/0510668.

[2]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[3]  J. M. Lewis,et al.  Dynamic Data Assimilation: A Least Squares Approach , 2006 .

[4]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[5]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[6]  Jeffrey L. Anderson.  Localization and Sampling Error Correction in Ensemble Kalman Filter Data Assimilation , 2012 .

[7]  Faming Liang,et al.  Statistical and Computational Inverse Problems , 2006, Technometrics.

[8]  A. Stordal,et al.  Bridging the ensemble Kalman filter and particle filters: the adaptive Gaussian mixture filter , 2011 .

[9]  Sebastian Reich,et al.  An ensemble Kalman-Bucy filter for continuous data assimilation , 2012 .

[10]  T. Lelièvre,et al.  Free Energy Computations: A Mathematical Perspective , 2010 .

[11]  James C. Robinson,et al.  Bayesian inverse problems for functions and applications to fluid mechanics , 2009 .

[12]  Peter J. Bickel,et al.  A Moment Matching Ensemble Filter for Nonlinear Non-Gaussian Data Assimilation , 2011 .

[13]  D. Crisan,et al.  Fundamentals of Stochastic Filtering , 2008 .

[14]  G. Evensen.  Data Assimilation: The Ensemble Kalman Filter , 2006 .

[15]  Gilbert Strang,et al.  Introduction to applied mathematics , 1988 .

[16]  Ernst Hairer,et al.  Numerical methods for evolutionary differential equations , 2010, Math. Comput..

[17]  Martin Verlaan,et al.  Conservation of Mass and Preservation of Positivity with Ensemble-Type Kalman Filter Algorithms , 2014 .

[18]  R. Handel,et al.  Can local particle filters beat the curse of dimensionality , 2013, 1301.6585.

[19]  Sebastian Reich,et al.  A Guided Sequential Monte Carlo Method for the Assimilation of Data into Stochastic Dynamical Systems , 2013 .

[20]  I. Olkin,et al.  The distance between two random vectors with given dispersion matrices , 1982 .

[21]  Andrew J. Majda,et al.  Filtering Complex Turbulent Systems , 2012 .

[22]  P. Moral.  Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[23]  Istvan Szunyogh,et al.  A Local Ensemble Kalman Filter for Atmospheric Data Assimilation , 2002 .

[24]  D. Pham.  Stochastic Methods for Sequential Data Assimilation in Strongly Nonlinear Systems , 2001 .

[25]  Dean S. Oliver,et al.  Conditioning Permeability Fields to Pressure Data , 1996 .

[26]  Istvan Szunyogh,et al.  Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter , 2005, physics/0511236.

[27]  Eugenia Kalnay,et al.  Accelerating the spin‐up of Ensemble Kalman Filtering , 2008, 0806.0180.

[28]  Sean P. Meyn,et al.  Feedback Particle Filter , 2013, IEEE Transactions on Automatic Control.

[29]  Peter Jan van Leeuwen,et al.  Efficient fully nonlinear data assimilation for geophysical fluid dynamics , 2013, Comput. Geosci..

[30]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[31]  M. Hp.  A class of markov processes associated with nonlinear parabolic equations. , 1966 .

[32]  P. Houtekamer,et al.  A Sequential Ensemble Kalman Filter for Atmospheric Data Assimilation , 2001 .

[33]  R. Caflisch.  Monte Carlo and quasi-Monte Carlo methods , 1998, Acta Numerica.

[34]  A. Stuart,et al.  MAP estimators and their consistency in Bayesian nonparametric inverse problems , 2013, 1303.4795.

[35]  Simo Srkk,et al.  Bayesian Filtering and Smoothing , 2013 .

[36]  Brian R. Hunt,et al.  Ensemble data assimilation for hyperbolic systems , 2013 .

[37]  A. Chorin,et al.  Implicit particle filtering for models with partial noise, and an application to geomagnetic data assimilation , 2011, 1109.3664.

[38]  P. Leeuwen,et al.  Nonlinear data assimilation in geosciences: an extremely efficient particle filter , 2010 .

[39]  G. Evensen,et al.  Analysis Scheme in the Ensemble Kalman Filter , 1998 .

[40]  C. Villani.  Topics in Optimal Transportation , 2003 .

[41]  A. Beskos,et al.  Error Bounds and Normalizing Constants for Sequential Monte Carlo in High Dimensions , 2011, 1112.1544.

[42]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[43]  R. McCann.  Existence and uniqueness of monotone measure-preserving maps , 1995 .

[44]  Youssef M. Marzouk,et al.  Bayesian inference with optimal maps , 2011, J. Comput. Phys..

[45]  Kayo Ide,et al.  Ensemble transform Kalman-Bucy filters , 2011 .

[46]  Sebastian Reich,et al.  A Gaussian‐mixture ensemble transform filter , 2011, 1102.3089.

[47]  S. Cohn,et al.  Ooce Note Series on Global Modeling and Data Assimilation Construction of Correlation Functions in Two and Three Dimensions and Convolution Covariance Functions , 2022 .

[48]  Dean S. Oliver,et al.  An Iterative EnKF for Strongly Nonlinear Systems , 2012 .

[49]  Sebastian Reich,et al.  A Nonparametric Ensemble Transform Method for Bayesian Inference , 2012, SIAM J. Sci. Comput..

[50]  A. Owen,et al.  Valuation of mortgage-backed securities using Brownian bridges to reduce effective dimension , 1997 .

[51]  C. Leith.  Theoretical Skill of Monte Carlo Forecasts , 1974 .

[52]  E. Süli,et al.  Numerical Solution of Partial Differential Equations , 2014 .

[53]  Simo Särkkä,et al.  Bayesian Filtering and Smoothing , 2013, Institute of Mathematical Statistics textbooks.

[54]  James C. Robinson.  Infinite-Dimensional Dynamical Systems: An Introduction to Dissipative Parabolic PDEs and the Theory of Global Attractors , 2001 .

[55]  Christopher K. Wikle,et al.  Atmospheric Modeling, Data Assimilation, and Predictability , 2005, Technometrics.

[56]  J. Whitaker,et al.  Ensemble Square Root Filters , 2003, Statistical Methods for Climate Scientists.

[57]  Hans R. Künsch,et al.  Mixture ensemble Kalman filters , 2013, Comput. Stat. Data Anal..

[58]  A. Beskos,et al.  Error Bounds and Normalising Constants for Sequential Monte Carlo Samplers in High Dimensions , 2014, Advances in Applied Probability.

[59]  Pierre Brasseur,et al.  A non-Gaussian analysis scheme using rank histograms for ensemble data assimilation , 2014 .

[60]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[61]  D. Nychka.  Data Assimilation” , 2006 .

[62]  R. Daley.  Atmospheric Data Analysis , 1991 .

[63]  J. Whitaker,et al.  Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter , 2001 .

[64]  D. Mayne,et al.  Monte Carlo techniques to estimate the conditional expectation in multi-stage non-linear filtering† , 1969 .

[65]  Lewis F. Richardson,et al.  Weather Prediction by Numerical Process , 1922 .

[66]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series, with Engineering Applications , 1949 .

[67]  Dean S. Oliver,et al.  On conditional simulation to inaccurate data , 1996 .

[68]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[69]  A. Stuart,et al.  Analysis of the 3DVAR filter for the partially observed Lorenz'63 model , 2012, 1212.4923.

[70]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[71]  Marc Bocquet,et al.  Combining inflation-free and iterative ensemble Kalman filters for strongly nonlinear systems , 2012 .

[72]  Michael Werman,et al.  Fast and robust Earth Mover's Distances , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[73]  H. McKean,et al.  A CLASS OF MARKOV PROCESSES ASSOCIATED WITH NONLINEAR PARABOLIC EQUATIONS , 1966, Proceedings of the National Academy of Sciences of the United States of America.

[74]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[75]  Matthias Morzfeld,et al.  A random map implementation of implicit filters , 2011, J. Comput. Phys..

[76]  Jens Schröter,et al.  A regulated localization scheme for ensemble‐based Kalman filters , 2012 .

[77]  Massimo Bonavita,et al.  On the use of EDA background error variances in the ECMWF 4D‐Var , 2012 .

[78]  G. Evensen.  Sequential data assimilation with a nonlinear quasi‐geostrophic model using Monte Carlo methods to forecast error statistics , 1994 .

[79]  A. Lorenc,et al.  Operational implementation of a hybrid ensemble/4D‐Var global data assimilation system at the Met Office , 2013 .

[80]  S. Julier,et al.  Which Is Better, an Ensemble of Positive–Negative Pairs or a Centered Spherical Simplex Ensemble? , 2004 .

[81]  Nancy Nichols,et al.  Unbiased ensemble square root filters , 2007 .

[82]  Jeffrey L. Anderson.  A Non-Gaussian Ensemble Filter Update for Data Assimilation , 2010 .

[83]  T. Miyoshi.  The Gaussian Approach to Adaptive Covariance Inflation and Its Implementation with the Local Ensemble Transform Kalman Filter , 2011 .

[84]  Jeffrey L. Anderson,et al.  An adaptive covariance inflation error correction algorithm for ensemble filters , 2007 .

[85]  P. Kitanidis.  Quasi‐Linear Geostatistical Theory for Inversing , 1995 .

[86]  Matthias Morzfeld,et al.  Implicit particle filters for data assimilation , 2010, 1005.4002.

[87]  P. L. Houtekamer,et al.  Ensemble Kalman filtering , 2005 .

[88]  S. Reich.  A dynamical systems framework for intermittent data assimilation , 2011 .

[89]  P. Bickel,et al.  Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems , 2008, 0805.3034.

[90]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[91]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[92]  S. Reich,et al.  A localization technique for ensemble Kalman filters , 2009, 0909.1678.

[93]  Colin J. Cotter,et al.  Ensemble filter techniques for intermittent data assimilation , 2013 .