Linearizing the Method of Conjugate Gradients by

The method of conjugate gradients (CG) is widely used for the iterative solution of large sparse systems of equations Ax = b, where A ∈ R is symmetric positive definite. Let xk denote the k–th iterate of CG. In this paper we obtain an expression for Jk, the Jacobian matrix of xk with respect to b. We use this expression to obtain computable bounds on the spectral norm condition number of xk, and to design algorithms to compute or estimate Jkv and J T k v for a given vector v. We also discuss several applications in which these ideas may be used. Numerical experiments are performed to illustrate the theory.

[1]  C. Lanczos.  An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[2]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[3]  A. Greenbaum.  Comparison of splittings used with the conjugate gradient algorithm , 1979 .

[4]  Gene H. Golub,et al.  Matrix computations , 1983 .

[5]  P. Hansen.  Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[6]  A. Greenbaum.  Behavior of slightly perturbed Lanczos and conjugate-gradient recurrences , 1989 .

[7]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[8]  Z. Strakos,et al.  On the real convergence rate of the conjugate gradient method , 1991 .

[9]  Anne Greenbaum,et al.  Predicting the Behavior of Finite Precision Lanczos and Conjugate Gradient Computations , 2015, SIAM J. Matrix Anal. Appl..

[10]  Henk A. van der Vorst,et al.  Approximate solutions and eigenvalue bounds from Krylov subspaces , 1995, Numer. Linear Algebra Appl..

[11]  B. Fischer.  Polynomial Based Iteration Methods for Symmetric Linear Systems , 1996 .

[12]  S. Godunov,et al.  Condition number of the Krylov bases and subspaces , 1996 .

[13]  Anne Greenbaum,et al.  Iterative methods for solving linear systems , 1997, Frontiers in applied mathematics.

[14]  Sergey V. Kuznetsov.  Perturbation bounds of the krylov bases and associated hessenberg forms , 1997 .

[15]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[16]  G. Meurant.  Computer Solution of Large Linear Systems , 1999 .

[17]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[18]  R. R. Hocking.  Methods and Applications of Linear Models: Regression and the Analysis of Variance , 2003 .

[19]  Ecmwf Newsletter,et al.  EUROPEAN CENTRE FOR MEDIUM-RANGE WEATHER FORECASTS , 2004 .

[20]  Lars Eldén,et al.  Partial least-squares vs. Lanczos bidiagonalization - I: analysis of a projection method for multiple regression , 2004, Comput. Stat. Data Anal..

[21]  G. Meurant.  The Lanczos and Conjugate Gradient Algorithms: From Theory to Finite Precision Computations , 2006 .

[22]  G. Meurant,et al.  The Lanczos and conjugate gradient algorithms in finite precision arithmetic , 2006, Acta Numerica.

[23]  Christopher C. Paige,et al.  A Useful Form of Unitary Matrix Obtained from Any Sequence of Unit 2-Norm n-Vectors , 2009, SIAM J. Matrix Anal. Appl..

[24]  Christopher C. Paige,et al.  An Augmented Stability Result for the Lanczos Hermitian Matrix Tridiagonalization Process , 2010, SIAM J. Matrix Anal. Appl..

[25]  Hernan G. Arango,et al.  Estimates of Analysis and Forecast Error Variances Derived from the Adjoint of 4D-Var , 2012 .