Data assimilation with the weighted ensemble Kalman filter

Abstract In this paper, two data assimilation methods based on sequential Monte Carlo sampling are studied and compared: the ensemble Kalman filter and the particle filter. Each of these techniques has its own advantages and drawbacks. In this work, we try to get the best of each method by combining them. The proposed algorithm, called the weighted ensemble Kalman filter, consists to rely on the EnsembleKalman Filter updates of samples in order to define a proposal distribution for the particle filter that depends on the history of measurement. The corresponding particle filter reveals to be efficient with a small number of samples and does not rely anymore on the Gaussian approximations of the ensemble Kalman filter. The efficiency of the new algorithm is demonstrated both in terms of accuracy and computational load. This latter aspect is of the utmost importance in meteorology or in oceanography since in these domains, data assimilation processes involve a huge number of state variables driven by highly non-linear dynamical models. Numerical experiments have been performed on different dynamical scenarios. The performances of the proposed technique have been compared to the ensemble Kalman filter procedure, which has demonstrated to provide meaningful results in geophysical sciences.

[1]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[2]  A. Lorenc.  A Global Three-Dimensional Multivariate Statistical Interpolation Scheme , 1981 .

[3]  F. L. Dimet,et al.  Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects , 1986 .

[4]  Andrew F. Bennett,et al.  Inverse Methods in Physical Oceanography: Frontmatter , 1992 .

[5]  A. Bennett.  Inverse Methods in Physical Oceanography , 1992 .

[6]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[7]  P. Courtier,et al.  Assimilation of Simulated Wind Lidar Data with a Kalman Filter , 1993 .

[8]  Jun S. Liu,et al.  Sequential Imputations and Bayesian Missing Data Problems , 1994 .

[9]  G. Evensen.  Sequential data assimilation with a nonlinear quasi‐geostrophic model using Monte Carlo methods to forecast error statistics , 1994 .

[10]  Philippe Courtier,et al.  Dual formulation of four‐dimensional variational assimilation , 1997 .

[11]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[12]  Andrew J. Majda,et al.  A Fourier-Wavelet Monte Carlo Method for Fractal Random Fields , 1997 .

[13]  P. Courtier,et al.  The ECMWF implementation of three‐dimensional variational assimilation (3D‐Var). II: Structure functions , 1998 .

[14]  G. Evensen,et al.  Analysis Scheme in the Ensemble Kalman Filter , 1998 .

[15]  Chi-Wang Shu.  Essentially non-oscillatory and weighted essentially non-oscillatory schemes for hyperbolic conservation laws , 1998 .

[16]  Centro internazionale matematico estivo. Session,et al.  Advanced Numerical Approximation of Nonlinear Hyperbolic Equations , 1998 .

[17]  P. Houtekamer,et al.  Data Assimilation Using an Ensemble Kalman Filter Technique , 1998 .

[18]  P. Courtier,et al.  The ECMWF implementation of three‐dimensional variational assimilation (3D‐Var). I: Formulation , 1998 .

[19]  Jeffrey L. Anderson,et al.  A Monte Carlo Implementation of the Nonlinear Filtering Problem to Produce Ensemble Assimilations and Forecasts , 1999 .

[20]  G. Evensen,et al.  An ensemble Kalman smoother for nonlinear dynamics , 2000 .

[21]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[22]  Nando de Freitas,et al.  The Unscented Particle Filter , 2000, NIPS.

[23]  T. Hamill,et al.  A Hybrid Ensemble Kalman Filter-3D Variational Analysis Scheme , 2000 .

[24]  Jeffrey L. Anderson.  An Ensemble Adjustment Kalman Filter for Data Assimilation , 2001 .

[25]  D. Pham.  Stochastic Methods for Sequential Data Assimilation in Strongly Nonlinear Systems , 2001 .

[26]  Craig H. Bishop,et al.  Adaptive sampling with the ensemble transform Kalman filter , 2001 .

[27]  P. Houtekamer,et al.  A Sequential Ensemble Kalman Filter for Atmospheric Data Assimilation , 2001 .

[28]  Brian R. Hunt,et al.  Identifying Low-Dimensional Nonlinear Behavior in Atmospheric Data , 2001 .

[29]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[30]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[31]  Christian Musso,et al.  Improving Regularised Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[32]  Arnaud Doucet,et al.  A survey of convergence results on particle filtering methods for practitioners , 2002, IEEE Trans. Signal Process..

[33]  Istvan Szunyogh,et al.  A Local Ensemble Kalman Filter for Atmospheric Data Assimilation , 2002 .

[34]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[35]  J. Whitaker,et al.  Ensemble Data Assimilation without Perturbed Observations , 2002 .

[36]  Peter Jan van Leeuwen.  Ensemble Kalman filters , Sequential Importance Resampling and beyond , 2003 .

[37]  P. J. van Leeuwen,et al.  A variance-minimizing filter for large-scale applications , 2003 .

[38]  J. Yorke,et al.  Four-dimensional ensemble Kalman filtering , 2004 .

[39]  Jeffrey L. Anderson.  A Local Least Squares Framework for Ensemble Filtering , 2003 .

[40]  G. Evensen,et al.  Sequential Data Assimilation Techniques in Oceanography , 2003 .

[41]  J. Whitaker,et al.  Ensemble Square Root Filters , 2003, Statistical Methods for Climate Scientists.

[42]  G. Kivman,et al.  Sequential parameter estimation for stochastic systems , 2003 .

[43]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[44]  Geir Evensen,et al.  The Ensemble Kalman Filter: theoretical formulation and practical implementation , 2003 .

[45]  P. Moral.  Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[46]  Istvan Szunyogh,et al.  A local ensemble Kalman filter for atmospheric data assimilation , 2004 .

[47]  R. Laubenfels,et al.  Feynman–Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2005 .

[48]  P. L. Houtekamer,et al.  Ensemble Kalman filtering , 2005 .

[49]  M. Zupanski.  Maximum Likelihood Ensemble Filter: Theoretical Aspects , 2005 .

[50]  M. Buehner,et al.  Atmospheric Data Assimilation with an Ensemble Kalman Filter: Results with Real Observations , 2005 .

[51]  Istvan Szunyogh,et al.  Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter , 2005, physics/0511236.

[52]  D. McLaughlin,et al.  Assessing the Performance of the Ensemble Kalman Filter for Land Surface Data Assimilation , 2006 .

[53]  G. Evensen.  Data Assimilation: The Ensemble Kalman Filter , 2006 .

[54]  Étienne Mémin,et al.  Partial Linear Gaussian Models for Tracking in Image Sequences Using Sequential Monte Carlo Methods , 2006, International Journal of Computer Vision.

[55]  Ionel Michael Navon,et al.  A Note on the Particle Filter with Posterior Gaussian Resampling , 2006 .

[56]  Istvan Szunyogh,et al.  Comparison between Local Ensemble Transform Kalman Filter and PSAS in the NASA finite volume GCM , 2006 .

[57]  B. Hunt,et al.  A comparative study of 4D-VAR and a 4D Ensemble Kalman Filter: perfect model simulations with Lorenz-96 , 2007 .

[58]  Takemasa Miyoshi,et al.  Local Ensemble Transform Kalman Filtering with an AGCM at a T159/L48 Resolution , 2007 .

[59]  Jeffrey L. Anderson,et al.  An adaptive covariance inflation error correction algorithm for ensemble filters , 2007 .

[60]  Brian R. Hunt,et al.  A non‐Gaussian Ensemble Filter for Assimilating Infrequent Noisy Observations , 2007 .

[61]  B. Hunt,et al.  Four-dimensional local ensemble transform Kalman filter: numerical experiments with a global circulation model , 2007 .

[62]  Istvan Szunyogh,et al.  Assessing Predictability with a Local Ensemble Kalman Filter , 2007 .

[63]  P. Bickel,et al.  Obstacles to High-Dimensional Particle Filtering , 2008 .

[64]  Istvan Szunyogh,et al.  A local ensemble transform Kalman filter data assimilation system for the NCEP global model , 2008 .

[65]  Thomas M. Hamill,et al.  Ensemble Data Assimilation with the NCEP Global Forecast System , 2008 .

[66]  T. Petti,et al.  Pressure image assimilation for atmospheric motion estimation , 2009 .

[67]  Peter Jan,et al.  Particle Filtering in Geophysical Systems , 2009 .

[68]  F. Gland,et al.  Large sample asymptotics for the ensemble Kalman filter , 2009 .

[69]  Anne Cuzol,et al.  A Stochastic Filtering Technique for Fluid Flow Velocity Fields Tracking , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  A. Majda,et al.  Catastrophic filter divergence in filtering nonlinear dissipative systems , 2010 .

[71]  Olivier Pannekoucke,et al.  Adaptation of a particle filtering method for data assimilation in a 1D numerical model used for fog forecasting , 2012 .