The ensemble Kalman filter for combined state and parameter estimation

This article provides a fundamental theoretical basis for understanding EnKF and serves as a useful text for future users. Data assimilation and parameter-estimation problems are explained, and the concept of joint parameter and state estimation, which can be solved using ensemble methods, is presented. KF and EKF are briefly discussed before introducing and deriving EnKF. Similarities and differences between KF and EnKF are pointed out. The benefits of using EnKF with high-dimensional and highly nonlinear dynamical models are illustrated by examples. EnKF and EnKS are also derived from Bayes theorem, using a probabilistic approach. The derivation is based on the assumption that measurement errors are independent in time and the model represents a Markov process, which allows for Bayes theorem to be written in a recursive form, where measurements are processed sequentially in time. The practical implementation of the analysis scheme isdiscussed, and it is shown that it can be computed efficiently in the space spanned by the ensemble realizations. The square root scheme is discussed as an alternative method that avoids the perturbation of measurements. However, the square root scheme has other pitfalls, and it is recommended to use the symmetric square root with or without a random rotation. The random rotation introduces a stochastic component to the update, and the quality of the scheme may then not improve compared to the original stochastic EnKF scheme with perturbed measurements.

[1]  R. E. Kalman,et al.  A New Approach to Linear Filtering and Prediction Problems , 2002 .

[2]  R. Kalman,et al.  New results in linear prediction and filtering theory Trans. AMSE , 1961 .

[3]  E. Lorenz.  Deterministic nonperiodic flow , 1963 .

[4]  F R HENDRICKSON,et al.  Which Is Better? , 2023, Advances in Cosmetic Surgery.

[5]  A. Jazwinski.  Stochastic Processes and Filtering Theory , 1970 .

[6]  G. Evensen.  Using the Extended Kalman Filter with a Multilayer Quasi-Geostrophic Ocean Model , 1992 .

[7]  Andrew F. Bennett,et al.  Inverse Methods in Physical Oceanography: Frontmatter , 1992 .

[8]  Andrew F. Bennett,et al.  Inverse Methods in Physical Oceanography: Bibliography , 1992 .

[9]  P. Courtier,et al.  Assimilation of Simulated Wind Lidar Data with a Kalman Filter , 1993 .

[10]  F. Bouttier.  A Dynamical Estimation of Forecast Error Covariances in an Assimilation System , 1994 .

[11]  P. Brasseur,et al.  Data Assimilation: Tools for Modelling the Ocean in a Global Change Perspective , 1994 .

[12]  Robert N. Miller.  Perspectives on Advanced Data Assimilation in Strongly Nonlinear Systems , 1994 .

[13]  G. Evensen.  Sequential data assimilation with a nonlinear quasi‐geostrophic model using Monte Carlo methods to forecast error statistics , 1994 .

[14]  Michael Ghil,et al.  Advanced data assimilation in strongly nonlinear dynamical systems , 1994 .

[15]  G. Evensen,et al.  Data assimilation and inverse methods in terms of a probabilistic formulation , 1996 .

[16]  Geir Evensen,et al.  Advanced Data Assimilation for Strongly Nonlinear Dynamics , 1997 .

[17]  G. Evensen,et al.  Parameter estimation solving a weak constraint variational formulation for an Ekman model , 1997 .

[18]  G. Evensen,et al.  Analysis Scheme in the Ensemble Kalman Filter , 1998 .

[19]  Jeffrey L. Anderson,et al.  A Monte Carlo Implementation of the Nonlinear Filtering Problem to Produce Ensemble Assimilations and Forecasts , 1999 .

[20]  G. Evensen,et al.  An ensemble Kalman smoother for nonlinear dynamics , 2000 .

[21]  Jeffrey L. Anderson.  An Ensemble Adjustment Kalman Filter for Data Assimilation , 2001 .

[22]  D. Pham.  Stochastic Methods for Sequential Data Assimilation in Strongly Nonlinear Systems , 2001 .

[23]  Craig H. Bishop,et al.  Adaptive sampling with the ensemble transform Kalman filter , 2001 .

[24]  P. Houtekamer,et al.  A Sequential Ensemble Kalman Filter for Atmospheric Data Assimilation , 2001 .

[25]  J. Whitaker,et al.  Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter , 2001 .

[26]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[27]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[28]  Istvan Szunyogh,et al.  A Local Ensemble Kalman Filter for Atmospheric Data Assimilation , 2002 .

[29]  D. Menemenlis.  Inverse Modeling of the Ocean and Atmosphere , 2002 .

[30]  Geir Evensen,et al.  Assimilation of SLA and SST data into an OGCM for the Indian Ocean , 2002 .

[31]  J. C. Muccino,et al.  Generalized inversion of the Korteweg-de Vries equation , 2002 .

[32]  J. Whitaker,et al.  Ensemble Data Assimilation without Perturbed Observations , 2002 .

[33]  M. Rienecker,et al.  Initial testing of a massively parallel ensemble Kalman filter with the Poseidon isopycnal ocean general circulation model , 2002 .

[34]  P. Houtekamer,et al.  Ensemble size, balance, and model-error representation in an ensemble Kalman filter , 2002 .

[35]  D. McLaughlin,et al.  Hydrologic Data Assimilation with the Ensemble Kalman Filter , 2002 .

[36]  Pierre Brasseur,et al.  A demonstration of ensemble-based assimilation methods with a layered OGCM from the perspective of operational ocean forecasting systems , 2003 .

[37]  Xuguang Wang,et al.  A Comparison of Breeding and Ensemble Transform Kalman Filter Ensemble Forecast Schemes , 2003 .

[38]  Christian L. Keppenne,et al.  Assimilation of temperature into an isopycnal ocean general circulation model using a parallel ensemble Kalman filter , 2003 .

[39]  Jeffrey L. Anderson.  A Local Least Squares Framework for Ensemble Filtering , 2003 .

[40]  J. Whitaker,et al.  Ensemble Square Root Filters , 2003, Statistical Methods for Climate Scientists.

[41]  Geir Evensen,et al.  The Ensemble Kalman Filter: theoretical formulation and practical implementation , 2003 .

[42]  Istvan Szunyogh,et al.  Assessing a local ensemble Kalman filter : perfect model experiments with the National Centers for Environmental Prediction global model , 2004 .

[43]  G. Evensen.  Sampling strategies and square root analysis schemes for the EnKF , 2004 .

[44]  S. Julier,et al.  Which Is Better, an Ensemble of Positive–Negative Pairs or a Centered Spherical Simplex Ensemble? , 2004 .

[45]  Istvan Szunyogh,et al.  A local ensemble Kalman filter for atmospheric data assimilation , 2004 .

[46]  Christian L. Keppenne,et al.  Ensemble Kalman filter assimilation of temperature and altimeter data with bias correction and application to seasonal prediction , 2005 .

[47]  Istvan Szunyogh,et al.  Assessing a local ensemble Kalman filter: perfect model experiments with the National Centers for Environmental Prediction global model , 2005 .

[48]  Emil Pelikán,et al.  An ensemble Kalman filter for short‐term forecasting of tropospheric ozone concentrations , 2005 .

[49]  P. L. Houtekamer,et al.  Ensemble Kalman filtering , 2005 .

[50]  Olwijn Leeuwenburgh,et al.  Assimilation of along‐track altimeter data in the tropical Pacific region of a global OGCM ensemble , 2005 .

[51]  Olwijn Leeuwenburgh,et al.  The impact of ensemble filter definition on the assimilation of temperature profiles in the tropical Pacific , 2005 .

[52]  Istvan Szunyogh,et al.  Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter , 2005, physics/0511236.

[53]  Dennis S. Bernstein,et al.  Data assimilation for magnetohydrodynamics systems , 2006 .

[54]  Arnaud Doucet,et al.  Sequential Monte Carlo Methods , 2006, Handbook of Graphical Models.

[55]  G. Evensen.  Data Assimilation: The Ensemble Kalman Filter , 2006 .

[56]  E. Kalnay,et al.  C ○ 2007 The Authors , 2006 .

[57]  B. Hunt,et al.  A comparative study of 4D-VAR and a 4D Ensemble Kalman Filter: perfect model simulations with Lorenz-96 , 2007 .

[58]  T. M. Chin,et al.  An Ensemble-Based Smoother with Retrospectively Updated Weights for Highly Nonlinear Systems , 2007 .

[59]  Nancy Nichols,et al.  Unbiased ensemble square root filters , 2007 .

[60]  P. Courtier,et al.  Variational Assimilation of Meteorological Observations With the Adjoint Vorticity Equation. I: Theory , 2007 .

[61]  Craig H. Bishop,et al.  Flow‐adaptive moderation of spurious ensemble correlations and its use in ensemble‐based data assimilation , 2007 .

[62]  Hilde Meisingset,et al.  Using the EnKF for Assisted History Matching of a North Sea Reservoir Model , 2007 .

[63]  Jeffrey L. Anderson,et al.  An adaptive covariance inflation error correction algorithm for ensemble filters , 2007 .

[64]  Istvan Szunyogh,et al.  Assimilating non-local observations with a local ensemble Kalman filter , 2007 .

[65]  Jeffrey L. Anderson.  EXPLORING THE NEED FOR LOCALIZATION IN ENSEMBLE DATA ASSIMILATION USING A HIERARCHICAL ENSEMBLE FILTER , 2007 .

[66]  Jeffrey L. Anderson,et al.  An investigation into the application of an ensemble Kalman smoother to high-dimensional geophysical systems , 2008 .

[67]  Istvan Szunyogh,et al.  A local ensemble transform Kalman filter data assimilation system for the NCEP global model , 2008 .

[68]  P. Oke,et al.  Implications of the Form of the Ensemble Transformation in the Ensemble Square Root Filters , 2008 .

[69]  Thomas M. Hamill,et al.  Ensemble Data Assimilation with the NCEP Global Forecast System , 2008 .

[70]  E. Kalnay,et al.  Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter , 2009 .