Limited‐memory preconditioners, with application to incremental four‐dimensional variational data assimilation

Incremental four-dimensional variational assimilation (4D-Var) is an algorithm that approximately solves a nonlinear minimization problem by solving a sequence of linearized (quadratic) minimization problems of the form where x is the control vector, A is a symmetric positive-definite matrix, b is a vector containing data and prior information, and c is a constant. This paper proposes a family of limited-memory preconditioners (LMPs) for accelerating the convergence of conjugate-gradient (CG) methods used to solve quadratic minimization problems such as those encountered in incremental 4D-Var. The family is constructed from a set of vectors {si : i = 1, …, l}, where each si is assumed to be conjugate with respect to the (Hessian) matrix A. In incremental 4D-Var, approximate LMPs from this family can be built using conjugate vectors generated during the CG minimization on previous outer iterations. The spectral and quasi-Newton LMPs employed in many operational 4D-Var systems are shown to be special cases of the family of LMPs proposed here. In addition, a new LMP based on Ritz vectors (approximate eigenvectors) is derived. The Ritz LMP can be interpreted as a stabilized version of the spectral LMP. Numerical experiments performed with a realistic global ocean 4D-Var system are presented, to test the impact of the three different preconditioners. The Ritz LMP is shown to be more effective than, or at least as effective as, the spectral and quasi-Newton LMPs in our 4D-Var experiments. Our experiments also demonstrate the importance of limiting the number of CG (inner) iterations on certain outer iterations to avoid possible divergence of the cost function on the outer loop. The optimal number of CG iterations will depend on the specific preconditioner used, and can be computed a priori, albeit at the expense of several evaluations of the cost function on the outer loop. In a cycled 4D-Var system, it may be necessary to perform this computation periodically to account for changes in the Hessian matrix arising from changes in the observing system and background-flow field. Copyright © 2008 Royal Meteorological Society

[1]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[2]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[3]  G. Meurant.  The Lanczos and Conjugate Gradient Algorithms: From Theory to Finite Precision Computations , 2006 .

[4]  Claude Lemaréchal,et al.  Some numerical experiments with variable-storage quasi-Newton algorithms , 1989, Math. Program..

[5]  Philippe Courtier,et al.  A New Hessian Preconditioning Method Applied to Variational Data Assimilation Experiments Using NASA General Circulation Models , 1996 .

[6]  Åke Björck,et al.  Numerical methods for least square problems , 1996 .

[7]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[8]  Jérôme Vialard,et al.  Three- and Four-Dimensional Variational Assimilation with a General Circulation Model of the Tropical Pacific Ocean. Part I: Formulation, Internal Diagnostics, and Consistency Checks , 2003 .

[9]  P. Courtier,et al.  A strategy for operational implementation of 4D‐Var, using an incremental approach , 1994 .

[10]  Serge Gratton,et al.  Approximate Gauss-Newton Methods for Nonlinear Least Squares Problems , 2007, SIAM J. Optim..

[11]  S. Ricci,et al.  A multivariate balance operator for variational ocean data assimilation , 2005 .

[12]  P. Courtier,et al.  Four-dimensional variational data assimilation using the adjoint of a multilevel primitive-equation model , 1991 .

[13]  Florence Rabier,et al.  Overview of global data assimilation developments in numerical weather‐prediction centres , 2005 .

[14]  J. Derber,et al.  A reformulation of the background error covariance in the ECMWF global data assimilation system , 1999 .

[15]  Philippe Courtier,et al.  Dual formulation of four‐dimensional variational assimilation , 1997 .

[16]  K. Chen,et al.  Matrix preconditioning techniques and applications , 2005 .

[17]  A. Sterl,et al.  The ERA‐40 re‐analysis , 2005 .

[18]  Nancy Nichols,et al.  Inner-Loop Stopping Criteria for Incremental Four-Dimensional Variational Data Assimilation , 2006 .

[19]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[20]  Å. Björck,et al.  Stability of Conjugate Gradient and Lanczos Methods for Linear Least Squares Problems , 1998, SIAM J. Matrix Anal. Appl..

[21]  J. Cullum,et al.  Lanczos algorithms for large symmetric eigenvalue computations , 1985 .

[22]  P. Toint,et al.  On Large Scale Nonlinear Least Squares Calculations , 1987 .

[23]  Yannick Trémolet.  Incremental 4D-Var convergence study , 2007 .

[24]  A. Simmons,et al.  The ECMWF operational implementation of four‐dimensional variational assimilation. I: Experimental results with simplified physics , 2007 .

[25]  Y. Trémolet.  Incremental 4D-Var convergence study , 2007 .

[26]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[27]  Hernan G. Arango,et al.  Weak and strong constraint data assimilation in the inverse Regional Ocean Modeling System (ROMS): Development and application for a baroclinic coastal upwelling system , 2007 .

[28]  M. Zupanski.  A Preconditioning Algorithm for Four-Dimensional Variational Data Assimilation , 1996 .

[29]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[30]  M. Zupanski,et al.  A preconditioning algorithm for large-scale minimization problems , 1993 .

[31]  Carl Tim Kelley,et al.  Iterative methods for optimization , 1999, Frontiers in applied mathematics.

[32]  Andrew C. Lorenc,et al.  Modelling of error covariances by 4D‐Var data assimilation , 2003 .

[33]  M. Benzi.  Preconditioning techniques for large linear systems: a survey , 2002 .

[34]  J. L. Rigal,et al.  On the Compatibility of a Given Solution With the Data of a Linear System , 1967, JACM.

[35]  M. Huddleston,et al.  Quality control of ocean temperature and salinity profiles — Historical and real-time data , 2007 .

[36]  Jorge Nocedal,et al.  Automatic Preconditioning by Limited Memory Quasi-Newton Updating , 1999, SIAM J. Optim..

[37]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[38]  Nancy Nichols,et al.  Approximate iterative methods for variational data assimilation , 2005 .

[39]  F. L. Dimet,et al.  Variational algorithms for analysis and assimilation of meteorological observations: theoretical aspects , 1986 .

[40]  Stephen J. Wright,et al.  Data assimilation in weather forecasting: a case study in PDE-constrained optimization , 2009 .