Accuracy of Two Three-term and Three Two-term Recurrences for Krylov Space Solvers

It has been widely observed that Krylov space solvers based on two three-term recurrences can give significantly less accurate residuals than mathematically equivalent solvers implemented with three two-term recurrences. In this paper we attempt to clarify and justify this difference theoretically by analyzing the gaps between recursively and explicitly computed residuals. It is shown that, in contrast with the two-term recurrences analyzed by Sleijpen, van der Vorst, and Fokkema [ Numer. Algorithms, 7 (1994), pp. 75--109] and Greenbaum [SIAM J. Matrix Anal. Appl., 18 (1997), pp. 535--551], in the two three-term recurrences the contributions of the local roundoff errors to the analyzed gaps may be dramatically amplified while propagating through the algorithm. This result explains, for example, the well-known behavior of three-term-based versions of the biconjugate gradient method, where large gaps between recursively and explicitly computed residuals are not uncommon. For the conjugate gradient method, however, such a devastating behavior---although possible---is not observed frequently in practical computations, and the difference between two-term and three-term implementations is usually moderate or small. This can also be explained by our results.

[1]  Martin H. Gutknecht,et al.  A matrix analysis of conjugate gradient algorithms , 1993 .

[2]  E. Stiefel,et al.  Relaxationsmethoden bester Strategie zur Lösung linearer Gleichungssysteme , 1955 .

[3]  J. Grcar.  Analyses of the lanczos algorithm and of the approximation problem in richardson's method , 1981 .

[4]  Gerard L. G. Sleijpen,et al.  The main effects of rounding errors in Krylov solvers for symmetric linear systems , 1997 .

[5]  T. Manteuffel,et al.  A taxonomy for conjugate gradient methods , 1990 .

[6]  H. Woźniakowski,et al.  Numerical stability of the Chebyshev method for the solution of large linear systems , 1977 .

[7]  Jo A. Bollen.  Numerical stability of descent methods for solving linear equations , 1984 .

[8]  C. Lanczos.  An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[9]  Kang C. Jea,et al.  Generalized conjugate-gradient acceleration of nonsymmetrizable iterative methods , 1980 .

[10]  Anne Greenbaum,et al.  Predicting the Behavior of Finite Precision Lanczos and Conjugate Gradient Computations , 2015, SIAM J. Matrix Anal. Appl..

[11]  H. Rutishauser.  Theory of Gradient Methods , 1959 .

[12]  H. Woźniakowski.  Round-off error analysis of iterations for large linear systems , 1978 .

[13]  Martin H. Gutknecht,et al.  Lanczos-type solvers for nonsymmetric linear systems of equations , 1997, Acta Numerica.

[14]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[15]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[16]  Henryk Woźniakowski.  Roundoff-error analysis of a new class of conjugate-gradient algorithms , 1980 .

[17]  A. Greenbaum.  Estimating the Attainable Accuracy of Recursively Computed Residual Methods , 1997, SIAM J. Matrix Anal. Appl..

[18]  T. Manteuffel.  The Tchebychev iteration for nonsymmetric linear systems , 1977 .

[19]  R. Freund,et al.  QMR: a quasi-minimal residual method for non-Hermitian linear systems , 1991 .

[20]  Roland W. Freund,et al.  An Implementation of the QMR Method Based on Coupled Two-Term Recurrences , 1994, SIAM J. Sci. Comput..

[21]  M. Rozložník,et al.  Numerical behaviour of the modified gram-schmidt GMRES implementation , 1997 .

[22]  N. Higham,et al.  COMPONENTWISE ERROR ANALYSIS FOR STATIONARY ITERATIVE METHODS , 1993 .

[23]  R. Fletcher.  Conjugate gradient methods for indefinite systems , 1976 .