A Stochastic Covariance Shrinkage Approach in Ensemble Transform Kalman Filtering

Operational Ensemble Kalman Filtering (EnKF) methods rely on model-specific heuristics such as localization, which are typically implemented based on the spacial locality of the model variables. We instead propose methods that more closely depend on the dynamics of the model, by looking at locally averaged-in-time behavior. Such behavior is typically described in terms of a climatological covariance of the dynamical system. We will be utilizing this covariance as the target matrix in covariance shrinkage methods, from which we will be drawing a synthetic ensemble in order to enrich both the ensemble transformation and the subspace of the ensemble.

[1]  Adrian Sandu,et al.  An Explicit Probabilistic Derivation of Inflation in a Scalar Ensemble Kalman Filter for Finite Step, Finite Ensemble Convergence , 2020, ArXiv.

[2]  Changhong Mou,et al.  Data-driven correction reduced order models for the quasi-geostrophic equations: a numerical investigation , 2019, International Journal of Computational Fluid Dynamics.

[3]  Adrian Sandu,et al.  ODE Test Problems: a MATLAB suite of initial value problems , 2019, ArXiv.

[4]  Adrian Sandu,et al.  A Bayesian Approach to Multivariate Adaptive Localization in Ensemble-Based Data Assimilation with Time-Dependent Extensions , 2018, Nonlinear Processes in Geophysics.

[5]  Xinwei Deng,et al.  An Ensemble Kalman Filter Implementation Based on Modified Cholesky Decomposition for Inverse Covariance Matrix Estimation , 2016, SIAM J. Sci. Comput..

[6]  Elias D. Nino-Ruiz,et al.  Efficient parallel implementation of DDDAS inference using an ensemble Kalman filter with shrinkage covariance matrix estimation , 2017, Cluster Computing.

[7]  Marc Bocquet,et al.  Data Assimilation: Methods, Algorithms, and Applications , 2016 .

[8]  Xinwei Deng,et al.  A parallel ensemble Kalman filter implementation based on modified Cholesky decomposition , 2015, ScalA '15.

[9]  Adrian Sandu,et al.  Ensemble Kalman filter implementations based on shrinkage covariance matrix estimation , 2015, Ocean Dynamics.

[10]  Colin J. Cotter,et al.  Probabilistic Forecasting and Bayesian Data Assimilation , 2015 .

[11]  A. Stuart,et al.  Data Assimilation: A Mathematical Introduction , 2015, 1506.07825.

[12]  Traian Iliescu,et al.  A stabilized proper orthogonal decomposition reduced-order model for large scale quasigeostrophic ocean circulation , 2014, Adv. Comput. Math..

[13]  Adrian Sandu,et al.  An efficient implementation of the ensemble Kalman filter based on an iterative Sherman–Morrison formula , 2013, Statistics and Computing.

[14]  Jeffrey L. Anderson Localization and Sampling Error Correction in Ensemble Kalman Filter Data Assimilation , 2012 .

[15]  Alfred O. Hero,et al.  Robust Shrinkage Estimation of High-Dimensional Covariance Matrices , 2010, IEEE Transactions on Signal Processing.

[16]  Dean S. Oliver,et al.  Ensemble filter methods with perturbed observations applied to nonlinear problems , 2010 .

[17]  Alfred O. Hero,et al.  Shrinkage Algorithms for MMSE Covariance Estimation , 2009, IEEE Transactions on Signal Processing.

[18]  Hisham Bin Zubair Efficient Multigrid Methods based on Improved Coarse Grid Correction Techniques , 2009 .

[19]  Natasha Flyer,et al.  A radial basis function method for the shallow water equations on a sphere , 2009, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[20]  Alfred O. Hero,et al.  Shrinkage estimation of high dimensional covariance matrices , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Jian Li,et al.  On Using a priori Knowledge in Space-Time Adaptive Processing , 2008, IEEE Transactions on Signal Processing.

[22]  P. Oke,et al.  Implications of the Form of the Ensemble Transformation in the Ensemble Square Root Filters , 2008 .

[23]  Ruth Petrie,et al.  Localization in the ensemble Kalman Filter , 2008 .

[24]  Peter R. Oke,et al.  A deterministic formulation of the ensemble Kalman filter : an alternative to ensemble square root filters , 2008 .

[25]  Jian Li,et al.  Knowledge-Aided Space-Time Adaptive Processing , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[26]  E. Lorenz Predictability of Weather and Climate: Predictability – a problem partly solved , 2006 .

[27]  G. Evensen Data Assimilation: The Ensemble Kalman Filter , 2006 .

[28]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[29]  Istvan Szunyogh,et al.  Efficient data assimilation for spatiotemporal chaos: A local ensemble transform Kalman filter , 2005, physics/0511236.

[30]  Christopher K. Wikle,et al.  Atmospheric Modeling, Data Assimilation, and Predictability , 2005, Technometrics.

[31]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[32]  M. Tribus,et al.  Probability theory: the logic of science , 2003 .

[33]  Colin B. Macdonald,et al.  Constructing high-order Runge-Kutta methods with embedded strong-stability-preserving pairs , 2003 .

[34]  Jeffrey L. Anderson An Ensemble Adjustment Kalman Filter for Data Assimilation , 2001 .

[35]  T. Hamill Interpretation of Rank Histograms for Verifying Ensemble Forecasts , 2001 .

[36]  Craig H. Bishop,et al.  Adaptive sampling with the ensemble transform Kalman filter , 2001 .

[37]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[38]  T. Hamill,et al.  A Hybrid Ensemble Kalman Filter-3D Variational Analysis Scheme , 2000 .

[39]  S. Cohn,et al.  Ooce Note Series on Global Modeling and Data Assimilation Construction of Correlation Functions in Two and Three Dimensions and Convolution Covariance Functions , 2022 .

[40]  G. Evensen,et al.  Analysis Scheme in the Ensemble Kalman Filter , 1998 .

[41]  M. Buhmann Radial functions on compact support , 1998, Proceedings of the Edinburgh Mathematical Society.

[42]  Francis X. Giraldo,et al.  Analysis of the Turkel-Zwas Scheme for the Two-Dimensional Shallow Water Equations in Spherical Coordinates , 1997 .

[43]  G. Evensen Sequential data assimilation with a nonlinear quasi‐geostrophic model using Monte Carlo methods to forecast error statistics , 1994 .

[44]  Ernst Hairer,et al.  Solving Ordinary Differential Equations I: Nonstiff Problems , 2009 .

[45]  L. Sirovich Turbulence and the dynamics of coherent structures. III. Dynamics and scaling , 1987 .

[46]  L. Sirovich Turbulence and the dynamics of coherent structures. I. Coherent structures , 1987 .

[47]  L. Sirovich Turbulence and the dynamics of coherent structures. II. Symmetries and transformations , 1987 .

[48]  D. Jespersen,et al.  Arakawa's method is a finite-element method , 1974 .

[49]  A. Arakawa Computational design for long-term numerical integration of the equations of fluid motion: two-dimen , 1997 .