An adaptive covariance inflation error correction algorithm for ensemble filters

ABSTRACT Ensemble filter methods for combining model prior estimates with observations of a system to produce improved posterior estimates of the system state are nowbeing applied to a wide range of problems both in and out of the geophysics community. Basic implementations of ensemble filters are simple to develop even without any data assimilation expertise. However, obtaining good performance using small ensembles and/or models with significant amounts of error can be more challenging.Anumber of adjunct algorithms have been developed to ameliorate errors in ensemble filters. The most common are covariance inflation and localization which have been used in many applications of ensemble filters. Inflation algorithms modify the prior ensemble estimates of the state variance to reduce filter error and avoid filter divergence. These adjunct algorithms can require considerable tuning for good performance, which can entail significant expense. A hierarchical Bayesian approach is used to develop an adaptive covariance inflation algorithm for use with ensemble filters. This adaptive error correction algorithm uses the same observations that are used to adjust the ensemble filter estimate of the state to estimate appropriate values of covariance inflation. Results are shown for several low-order model examples and the algorithm produces results that are comparable with the best tuned inflation values, even for small ensembles in the presence of very large model error.

[1]  Stephen J. Lord,et al.  The New Global Operational Analysis System at the National Meteorological Center , 1991 .

[2]  R. Todling,et al.  Data Assimilation in the Presence of Forecast Bias: The GEOS Moisture Analysis , 2000 .

[3]  J. Derber.  A Variational Continuous Assimilation Technique , 1989 .

[4]  M. Buehner,et al.  Atmospheric Data Assimilation with an Ensemble Kalman Filter: Results with Real Observations , 2005 .

[5]  D. Pham.  Stochastic Methods for Sequential Data Assimilation in Strongly Nonlinear Systems , 2001 .

[6]  E. Lorenz.  Predictability of Weather and Climate: Predictability – a problem partly solved , 2006 .

[7]  A. Sterl,et al.  The ERA‐40 re‐analysis , 2005 .

[8]  P. Houtekamer,et al.  A Sequential Ensemble Kalman Filter for Atmospheric Data Assimilation , 2001 .

[9]  J. R. Eyre,et al.  Assimilation of TOVS radiance information through one-dimensional variational analysis , 1993 .

[10]  Istvan Szunyogh,et al.  A Local Ensemble Kalman Filter for Atmospheric Data Assimilation , 2002 .

[11]  T. Bengtsson,et al.  Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants , 2007 .

[12]  M. Rienecker,et al.  Initial testing of a massively parallel ensemble Kalman filter with the Poseidon isopycnal ocean general circulation model , 2002 .

[13]  T. Palmer,et al.  Stochastic representation of model uncertainties in the ECMWF ensemble prediction system , 2007 .

[14]  S. Cohn,et al.  Ooce Note Series on Global Modeling and Data Assimilation Construction of Correlation Functions in Two and Three Dimensions and Convolution Covariance Functions , 2022 .

[15]  G. Evensen,et al.  Analysis Scheme in the Ensemble Kalman Filter , 1998 .

[16]  P. Courtier,et al.  Variational Assimilation of Meteorological Observations With the Adjoint Vorticity Equation. Ii: Numerical Results , 2007 .

[17]  James A. Hansen.  Accounting for Model Error in Ensemble-Based State Estimation and Forecasting , 2002 .

[18]  P. Comba,et al.  Part I. Theory , 2007 .

[19]  A. Simmons,et al.  The ECMWF operational implementation of four‐dimensional variational assimilation. I: Experimental results with simplified physics , 2007 .

[20]  Jeffrey L. Anderson.  EXPLORING THE NEED FOR LOCALIZATION IN ENSEMBLE DATA ASSIMILATION USING A HIERARCHICAL ENSEMBLE FILTER , 2007 .

[21]  R. Daley.  Estimating observation error statistics for atmospheric data assimilation , 1993 .

[22]  Andrew C. Lorenc,et al.  The potential of the ensemble Kalman filter for NWP—a comparison with 4D‐Var , 2003 .

[23]  D. Dee.  On-line Estimation of Error Covariance Parameters for Atmospheric Data Assimilation , 1995 .

[24]  Ionel M. Navon,et al.  Optimality of variational data assimilation and its relationship with the Kalman filter and smoother , 2001 .

[25]  P. Courtier,et al.  A strategy for operational implementation of 4D‐Var, using an incremental approach , 1994 .

[26]  J. Whitaker,et al.  Ensemble Data Assimilation without Perturbed Observations , 2002 .

[27]  P. Houtekamer,et al.  Data Assimilation Using an Ensemble Kalman Filter Technique , 1998 .

[28]  W. Collins,et al.  The NCEP–NCAR 50-Year Reanalysis: Monthly Means CD-ROM and Documentation , 2001 .

[29]  K. Emanuel,et al.  Optimal Sites for Supplementary Weather Observations: Simulation with a Small Model , 1998 .

[30]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[31]  Dick Dee,et al.  Maximum-Likelihood Estimation of Forecast and Observation Error Covariance Parameters. Part II: Applications , 1999 .

[32]  Dick Dee,et al.  Ooce Note Series on Global Modeling and Data Assimilation Maximum-likelihood Estimation of Forecast and Observation Error Covariance Parameters , 2022 .

[33]  Thomas Slawig,et al.  Generating efficient derivative code with TAF: Adjoint and tangent linear Euler flow around an airfoil , 2005, Future Gener. Comput. Syst..

[34]  J. Mahfouf,et al.  The ecmwf operational implementation of four‐dimensional variational assimilation. III: Experimental results and diagnostics with operational configuration , 2000 .

[35]  G. Evensen.  Sequential data assimilation with a nonlinear quasi‐geostrophic model using Monte Carlo methods to forecast error statistics , 1994 .

[36]  J. Whitaker,et al.  Ensemble Square Root Filters , 2003, Statistical Methods for Climate Scientists.

[37]  J. Whitaker,et al.  Accounting for the Error due to Unresolved Scales in Ensemble Data Assimilation: A Comparison of Different Approaches , 2005 .

[38]  Jeffrey L. Anderson,et al.  A Monte Carlo Implementation of the Nonlinear Filtering Problem to Produce Ensemble Assimilations and Forecasts , 1999 .

[39]  P. Houtekamer,et al.  Ensemble size, balance, and model-error representation in an ensemble Kalman filter , 2002 .

[40]  P. Courtier,et al.  Variational Assimilation of Meteorological Observations With the Adjoint Vorticity Equation. I: Theory , 2007 .

[41]  L. Isaksen,et al.  The ERA-40 Reanalysis , 2004 .

[42]  Jeffrey L. Anderson.  An Ensemble Adjustment Kalman Filter for Data Assimilation , 2001 .

[43]  Dick Dee,et al.  Maximum-Likelihood Estimation of Forecast and Observation Error Covariance Parameters. Part I: Methodology , 1999 .

[44]  J. Szpunar,et al.  Part II applications , 2003 .

[45]  J. Whitaker,et al.  Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter , 2001 .

[46]  Jeffrey L. Anderson.  A Local Least Squares Framework for Ensemble Filtering , 2003 .

[47]  G. Mellor,et al.  Development of a turbulence closure model for geophysical fluid problems , 1982 .

[48]  P. Houtekamer,et al.  An Adaptive Ensemble Kalman Filter , 2000 .

[49]  F. L. Dimet,et al.  Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects , 1986 .