LARGE-SCALE KALMAN FILTERING USING THE LIMITED MEMORY BFGS METHOD

The standard formulations of the Kalman filter (KF) and exten ded Kalman filter (EKF) require the storage and multiplication of matrices of size , where is the size of the state space, and the inversion of matrices of size , where is the size of the observation space. Thus when both and are large, implementation issues arise. In this paper, we advocate the use of the limited memory BFGS method (LBFGS) to address these issues. A detailed description of how to use LBFGS within both the KF and EKF methods is given. The methodology is then tested on two examples: the first is la rge-scale and linear, and the second is small scale and nonlinear. Our results indicate that the resulting methods , which we will denote LBFGS-KF and LBFGS-EKF, yield results that are comparable with those obtained using KF and EKF, respectively, and can be used on much larger scale problems.

[1]  Alexey Kaplan,et al.  Mapping tropical Pacific sea level : Data assimilation via a reduced state space Kalman filter , 1996 .

[2]  E. Lorenz.  Predictability of Weather and Climate: Predictability – a problem partly solved , 2006 .

[3]  Matthew Fisher,et al.  On the equivalence between Kalman smoothing and weak‐constraint four‐dimensional variational data assimilation , 2005, Quarterly Journal of the Royal Meteorological Society.

[4]  Xiangjun Tian,et al.  An ensemble-based explicit four-dimensional variational assimilation method , 2008 .

[5]  Philippe Courtier,et al.  A New Hessian Preconditioning Method Applied to Variational Data Assimilation Experiments Using NASA General Circulation Models , 1996 .

[6]  J. Mahfouf,et al.  The ecmwf operational implementation of four‐dimensional variational assimilation. III: Experimental results and diagnostics with operational configuration , 2000 .

[7]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[8]  Clive D Rodgers,et al.  Inverse Methods for Atmospheric Sounding: Theory and Practice , 2000 .

[9]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[10]  Serge Gratton,et al.  Limited‐memory preconditioners, with application to incremental four‐dimensional variational data assimilation , 2008 .

[11]  Didier Auroux,et al.  Limited-Memory BFGS Diagonal Preconditioners for a Data Assimilation Problem in Meteorology , 2000 .

[12]  K. Emanuel,et al.  Optimal Sites for Supplementary Weather Observations: Simulation with a Small Model , 1998 .

[13]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[14]  Fabrice Veersé,et al.  Variable-storage Quasi-Newton Operators as Inverse Forecast/Analysis Error Covariance Matrices in Variational Data Assimilation , 1999 .

[15]  Ecmwf Newsletter,et al.  EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS , 2004 .

[16]  L. Nazareth.  Relationship between the BFGS and conjugate gradient algorithms , 1977 .

[17]  M. Fisher,et al.  347 Developments in 4 D-Var and Kalman Filtering , 1994 .

[18]  Andrew C. Lorenc,et al.  Modelling of error covariances by 4D‐Var data assimilation , 2003 .

[19]  Heikki Haario,et al.  OPTIMAL APPROXIMATION OF KALMAN FILTERING WITH TEMPORALLY LOCAL 4D-VAR IN OPERATIONAL WEATHER FORECASTING , 2005 .

[20]  R. Daley.  Atmospheric Data Analysis , 1991 .

[21]  Heikki Haario,et al.  The variational Kalman filter and an efficient implementation using limited memory BFGS , 2010 .

[22]  Ionel M. Navon,et al.  A reduced‐order approach to four‐dimensional variational data assimilation using proper orthogonal decomposition , 2007 .

[23]  F. L. Dimet,et al.  Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects , 1986 .

[24]  Stephen J. Wright,et al.  Data assimilation in weather forecasting: a case study in PDE-constrained optimization , 2009 .

[25]  Ionel M. Navon,et al.  Optimality of variational data assimilation and its relationship with the Kalman filter and smoother , 2001 .

[26]  D. Dee.  Simplification of the Kalman filter for meteorological data assimilation , 1991 .

[27]  A. Simmons,et al.  The ECMWF operational implementation of four‐dimensional variational assimilation. I: Experimental results with simplified physics , 2007 .

[28]  I. Yu. Gejadze,et al.  On Analysis Error Covariances in Variational Data Assimilation , 2008, SIAM J. Sci. Comput..

[29]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[30]  Arto Voutilainen,et al.  A filtering approach for estimating lake water quality from remote sensing data , 2007, Int. J. Appl. Earth Obs. Geoinformation.

[31]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[32]  Martin Leutbecher.  A data assimilation tutorial based on the Lorenz-95 system , 2010 .