Unscented/ensemble transform-based variational filter

Abstract To deal with high dimensional nonlinear filtering problems, a hybrid scheme called Unscented/Ensemble transform Variational Filter (UEVF) is introduced. This is the combination of an Unscented Transform (UT), an Ensemble Transform (ET) and a rank-reduction method to compute the background covariance error matrices as well as a variational minimization to conduct the mean correction. The proposed UEVF is more efficient than the Unscented Kalman Filter (UKF) to estimate the ensemble mean and covariance by the blending of a variational optimization instead of a Kalman linear correction as well as the ET-like covariance estimation into the update step. Moreover, in order to tackle the high dimension dynamics, truncated singular value decomposition is applied to provide a size reduction of a sigma-points set with an adaptive fashion. For performance verifications, we present two numerical experiments with different dynamics. The first system is the chaotic and high dimensional Lorenz-95 model. We show the performance of different filters including the UEVF as the increasing of dimensionality or noise level. The second simulation is a model based on the 2D shallow water equation. The same tests are provided on this hydrodynamical system. All the numerical experiments confirm that the UEVF outperforms the widely applied Kalman-like filters explicitly.

[1]  Philip E. Gill,et al.  Practical optimization , 1981 .

[2]  I. M. Moroz,et al.  Ensemble Kalman filter with the unscented transform , 2009, 0901.0461.

[3]  F. Gland,et al.  Large sample asymptotics for the ensemble Kalman filter , 2009 .

[4]  G. Evensen.  Sequential data assimilation with a nonlinear quasi‐geostrophic model using Monte Carlo methods to forecast error statistics , 1994 .

[5]  Pavel Sakov.  Comment on “Ensemble Kalman filter with the unscented transform” , 2009 .

[6]  Rudolph van der Merwe,et al.  The square-root unscented Kalman filter for state and parameter-estimation , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[7]  Ionel Michael Navon,et al.  Adaptive ensemble reduction and inflation , 2007 .

[8]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[9]  S. Julier,et al.  Which Is Better, an Ensemble of Positive–Negative Pairs or a Centered Spherical Simplex Ensemble? , 2004 .

[10]  Nancy Nichols,et al.  Unbiased ensemble square root filters , 2007 .

[11]  J. Whitaker,et al.  Ensemble Square Root Filters , 2003, Statistical Methods for Climate Scientists.

[12]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[13]  P. Hansen.  The truncatedSVD as a method for regularization , 1987 .

[14]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[15]  P. Moral.  Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[16]  J. Whitaker,et al.  Ensemble Data Assimilation without Perturbed Observations , 2002 .

[17]  G. Evensen,et al.  Asynchronous data assimilation with the EnKF , 2010 .

[18]  K. Emanuel,et al.  Optimal Sites for Supplementary Weather Observations: Simulation with a Small Model , 1998 .

[19]  P. Houtekamer,et al.  A Sequential Ensemble Kalman Filter for Atmospheric Data Assimilation , 2001 .

[20]  Mark A. Liniger,et al.  The discrete brier and ranked probability skill scores , 2007 .

[21]  Magnus R. Hestenes,et al.  Conjugate Direction Methods in Optimization , 1980 .

[22]  Jeffrey K. Uhlmann,et al.  Unscented filtering and nonlinear estimation , 2004, Proceedings of the IEEE.

[23]  T. Kailath,et al.  A state-space approach to adaptive RLS filtering , 1994, IEEE Signal Processing Magazine.

[24]  E. Lorenz.  Predictability of Weather and Climate: Predictability – a problem partly solved , 2006 .

[25]  Irene M. Moroz,et al.  Reply to “Comment on ‘Ensemble Kalman filter with the unscented transform”’ , 2010 .

[26]  Ionel M. Navon,et al.  Conjugate-Gradient Methods for Large-Scale Minimization in Meteorology , 1987 .

[27]  P. Houtekamer,et al.  Data Assimilation Using an Ensemble Kalman Filter Technique , 1998 .

[28]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[29]  R. Daley.  Atmospheric Data Analysis , 1991 .

[30]  J. Whitaker,et al.  Distance-dependent filtering of background error covariance estimates in an ensemble Kalman filter , 2001 .

[31]  P. Ioannou,et al.  State Estimation Using a Reduced-Order Kalman Filter , 2001 .

[32]  Geir Evensen,et al.  The Ensemble Kalman Filter: theoretical formulation and practical implementation , 2003 .

[33]  G. Evensen.  The ensemble Kalman filter for combined state and parameter estimation , 2009, IEEE Control Systems.

[34]  Andrew C. Lorenc,et al.  Analysis methods for numerical weather prediction , 1986 .

[35]  P. Malanotte‐Rizzoli,et al.  Reduced‐rank Kalman filters applied to an idealized model of the wind‐driven ocean circulation , 2003 .

[36]  N. Papadakis,et al.  Data assimilation with the weighted ensemble Kalman filter , 2010 .

[37]  Jeffrey L. Anderson.  An Ensemble Adjustment Kalman Filter for Data Assimilation , 2001 .

[38]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[39]  P. Courtier,et al.  Variational assimilation of meteorological observations with the direct and adjoint shallow-water equations , 1990 .

[40]  B. Moore.  Principal component analysis in linear systems: Controllability, observability, and model reduction , 1981 .

[41]  Nando de Freitas,et al.  The Unscented Particle Filter , 2000, NIPS.

[42]  HansenPer Christian.  The truncated SVD as a method for regularization , 1987 .

[43]  Craig H. Bishop,et al.  Adaptive sampling with the ensemble transform Kalman filter , 2001 .

[44]  Hugh F. Durrant-Whyte,et al.  A new method for the nonlinear transformation of means and covariances in filters and estimators , 2000, IEEE Trans. Autom. Control..

[45]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[46]  T. Hamill,et al.  A Hybrid Ensemble Kalman Filter-3D Variational Analysis Scheme , 2000 .

[47]  Martin Ehrendorfer,et al.  Optimal Prediction of Forecast Error Covariances through Singular Vectors , 1997 .

[48]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[49]  P. Courtier,et al.  The ECMWF implementation of three‐dimensional variational assimilation (3D‐Var). I: Formulation , 1998 .

[50]  Fathalla A. Rihan,et al.  Assimilation of Doppler radial winds into a 3D‐Var system: Errors and impact of radial velocities on the variational analysis and model forecasts , 2008 .

[51]  P. Lax,et al.  Systems of conservation laws , 1960 .

[52]  Peter R. Oke,et al.  Ensemble member generation for sequential data assimilation , 2008 .