On A Class of Limited Memory Preconditioners For Large Scale Linear Systems With Multiple Right-Hand Sides

This work studies a class of limited memory preconditioners (LMPs) for solving linear (positive-definite) systems of equations with multiple right-hand sides. We propose a class of (LMPs), whose construction requires a small number of linearly independent vectors. After exploring the theoretical properties of the preconditioners, we focus on three particular members: spectral-LMP, quasi-Newton-LMP, and Ritz-LMP. We show that the first two are well known, while the third is new. Numerical tests indicate that the Ritz-LMP is efficient on a real-life nonlinear optimization problem arising in a data assimilation system for oceanography.

[1]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[2]  Cornelis Vuik,et al.  A Comparison of Deflation and Coarse Grid Correction Applied to Porous Media Flow , 2004, SIAM J. Numer. Anal..

[3]  Lois Mansfield.  On the conjugate gradient solution of the Schur complement system obtained from domain decomposition , 1990 .

[4]  K. Chen,et al.  Matrix preconditioning techniques and applications , 2005 .

[5]  Owe Axelsson.  OPTIMAL PRECONDITIONERS BASED ON RATE OF CONVERGENCE ESTIMATES FOR THE CONJUGATE GRADIENT METHOD , 2001 .

[6]  J. Mandel.  Balancing domain decomposition , 1993 .

[7]  Andrew C. Lorenc,et al.  Modelling of error covariances by 4D‐Var data assimilation , 2003 .

[8]  Jorge Nocedal,et al.  Automatic Preconditioning by Limited Memory Quasi-Newton Updating , 1999, SIAM J. Optim..

[9]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[10]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[11]  Ilse C. F. Ipsen,et al.  THE IDEA BEHIND KRYLOV METHODS , 1998 .

[12]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[13]  Luc Giraud,et al.  On the Sensitivity of Some Spectral Preconditioners , 2005, SIAM J. Matrix Anal. Appl..

[14]  S. Gratton,et al.  Incremental spectral preconditioners for sequences of linear systems , 2007 .

[15]  S. Ricci,et al.  A multivariate balance operator for variational ocean data assimilation , 2005 .

[16]  Gene H. Golub,et al.  Matrix computations , 1983 .

[17]  Owe Axelsson,et al.  Generalized Augmented Matrix Preconditioning Approach and its Application to Iterative Solution of Ill-Conditioned Algebraic Systems , 2000, SIAM J. Matrix Anal. Appl..

[18]  Serge Gratton,et al.  Limited‐memory preconditioners, with application to incremental four‐dimensional variational data assimilation , 2008 .

[19]  Jinchao Xu,et al.  Iterative Methods by Space Decomposition and Subspace Correction , 1992, SIAM Rev..

[20]  J. Derber,et al.  A reformulation of the background error covariance in the ECMWF global data assimilation system , 1999 .

[21]  Lois Mansfield.  Damped Jacobi Preconditioning and Coarse Grid Deflation for Conjugate Gradient Iteration on Parallel Computers , 1991, SIAM J. Sci. Comput..

[22]  Cornelis Vuik,et al.  A Comparison of Deflation and the Balancing Preconditioner , 2005, SIAM J. Sci. Comput..

[23]  L. Nazareth.  A Relationship between the BFGS and Conjugate Gradient Algorithms and Its Implications for New Algorithms , 1979 .

[24]  Stephen J. Wright,et al.  Data assimilation in weather forecasting: a case study in PDE-constrained optimization , 2009 .

[25]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[26]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[27]  Bruno Carpentieri,et al.  A Class of Spectral Two-Level Preconditioners , 2003, SIAM J. Sci. Comput..

[28]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[29]  R. Schnabel.  Quasi-Newton Methods Using Multiple Secant Equations. , 1983 .

[30]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[31]  I. Duff,et al.  The state of the art in numerical analysis , 1997 .

[32]  Jérôme Vialard,et al.  Three- and Four-Dimensional Variational Assimilation with a General Circulation Model of the Tropical Pacific Ocean. Part I: Formulation, Internal Diagnostics, and Consistency Checks , 2003 .

[33]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[34]  Å. Björck,et al.  Stability of Conjugate Gradient and Lanczos Methods for Linear Least Squares Problems , 1998, SIAM J. Matrix Anal. Appl..

[35]  Haim Waisman,et al.  The generalized global basis (GGB) method , 2004 .

[36]  Nicholas I. M. Gould,et al.  Methods for nonlinear constraints in optimization calculations , 1996 .