CG Versus MINRES: An Empirical Comparison

For iterative solution of symmetric systems  the conjugate gradient method (CG) is commonly used when A is positive definite, while the minimum residual method (MINRES) is typically reserved for indefinite systems. We investigate the sequence of approximate solutions  generated by each method and suggest that even if A is positive definite, MINRES may be preferable to CG if iterations are to be terminated early. In particular, we show for MINRES that the solution norms  are monotonically increasing when A is positive definite (as was already known for CG), and the solution errors  are monotonically decreasing. We also show that the backward errors for the MINRES iterates  are monotonically decreasing.

[1]  C. Lanczos.  An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[2]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[3]  E. Stiefel,et al.  Relaxationsmethoden bester Strategie zur Lösung linearer Gleichungssysteme , 1955 .

[4]  D. Luenberger.  Hyperbolic Pairs in the Method of Conjugate Gradients , 1969 .

[5]  D. Luenberger.  The Conjugate Residual Method for Constrained Minimization Problems , 1970 .

[6]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[7]  G. W. Stewart,et al.  Research, Development, and LINPACK , 1977 .

[8]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[9]  Michael A. Saunders,et al.  Algorithm 583: LSQR: Sparse Linear Equations and Least Squares Problems , 1982, TOMS.

[10]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[11]  Jack Dongarra,et al.  LINPACK Users' Guide , 1987 .

[12]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[13]  David S. Watkins,et al.  Fundamentals of matrix computations , 1991 .

[14]  Louette R. Johnson Lutjens.  Research , 2006 .

[15]  G. Golub,et al.  Iterative solution of linear systems , 1991, Acta Numerica.

[16]  Anne Greenbaum,et al.  Iterative methods for solving linear systems , 1997, Frontiers in applied mathematics.

[17]  Yong Sun.  The filter algorithm for solving large-scale eigenproblems from accelerator simulations , 2003 .

[18]  M. Arioli,et al.  A stopping criterion for the conjugate gradient algorithm in a finite element method framework , 2000, Numerische Mathematik.

[19]  G. Meurant.  The Lanczos and Conjugate Gradient Algorithms: From Theory to Finite Precision Computations , 2006 .

[20]  G. Meurant.  The Lanczos and conjugate gradient algorithms , 2008 .

[21]  Michael A. Saunders,et al.  LSMR: An Iterative Algorithm for Sparse Least-Squares Problems , 2010, SIAM J. Sci. Comput..

[22]  D. Titley-Péloquin.  Backward perturbation analysis of least squares problems , 2010 .

[23]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[24]  Michael A. Saunders,et al.  MINRES-QLP: A Krylov Subspace Method for Indefinite or Singular Symmetric Systems , 2010, SIAM J. Sci. Comput..