MINRES-QLP: A Krylov Subspace Method for Indefinite or Singular Symmetric Systems

CG, SYMMLQ, and MINRES are Krylov subspace methods for solving symmetric systems of linear equations. When these methods are applied to an incompatible system (that is, a singular symmetric least-squares problem), CG could break down and SYMMLQ's solution could explode, while MINRES would give a least-squares solution but not necessarily the minimum-length (pseudoinverse) solution. This understanding motivates us to design a MINRES-like algorithm to compute minimum-length solutions to singular symmetric systems. MINRES uses QR factors of the tridiagonal matrix from the Lanczos process (where $R$ is upper-tridiagonal). MINRES-QLP uses a QLP decomposition (where rotations on the right reduce $R$ to lower-tridiagonal form). On ill-conditioned systems (singular or not), MINRES-QLP can give more accurate solutions than MINRES. We derive preconditioned MINRES-QLP, new stopping rules, and better estimates of the solution and residual norms, the matrix norm, and the condition number.

[1]  J. Hadamard.  Sur les problemes aux derive espartielles et leur signification physique , 1902 .

[2]  C. Lanczos.  An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[3]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[4]  G. Stewart.  On the Continuity of the Generalized Inverse , 1969 .

[5]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[6]  C. Paige.  Error Analysis of the Lanczos Algorithm for Tridiagonalizing a Symmetric Matrix , 1976 .

[7]  G. W. Stewart,et al.  Research, Development, and LINPACK , 1977 .

[8]  L. Eldén.  Algorithms for the regularization of ill-conditioned least squares problems , 1977 .

[9]  J. G. Lewis.  Algorithms for sparse matrix eigenvalue problems , 1977 .

[10]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[11]  Michael A. Saunders,et al.  Algorithm 583: LSQR: Sparse Linear Equations and Least Squares Problems , 1982, TOMS.

[12]  Gene H. Golub,et al.  Matrix computations , 1983 .

[13]  P. Strevens.  Iii , 1985 .

[14]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[15]  E. F. Kaasschieter,et al.  Preconditioned conjugate gradients for solving singular systems , 1988 .

[16]  Per Christian Hansen,et al.  Truncated Singular Value Decomposition Solutions to Discrete Ill-Posed Problems with Ill-Determined Numerical Rank , 1990, SIAM J. Sci. Comput..

[17]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[18]  Y. Notay.  Solving positive (semi) definite linear systems by preconditioned iterative methods , 1991 .

[19]  Louette R. Johnson Lutjens.  Research , 2006 .

[20]  Michael A. Saunders,et al.  Preconditioners for Indefinite Systems Arising in Optimization , 1992, SIAM J. Matrix Anal. Appl..

[21]  Dianne P. O'Leary,et al.  The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems , 1993, SIAM J. Sci. Comput..

[22]  R. Freund,et al.  A new Krylov-subspace method for symmetric indefinite linear systems , 1994 .

[23]  M. Saunders.  Solution of sparse rectangular systems using LSQR and CRAIG , 1995 .

[24]  P. Dooren,et al.  High performance algorithms for Toeplitz and block Toeplitz matrices , 1996 .

[25]  J. Nagy,et al.  Restoration of atmospherically blurred images by symmetric indefinite conjugate gradient techniques , 1996 .

[26]  M. Saunders.  Computing projections with LSQR , 1997 .

[27]  L. Trefethen,et al.  Numerical linear algebra , 1997 .

[28]  Franciszek A. Dul,et al.  MINRES and MINERR Are Better than SYMMLQ in Eigenpair Computations , 1998, SIAM J. Sci. Comput..

[29]  A. Wathen,et al.  Minimum residual methods for augmented systems , 1998 .

[30]  Panayot S. Vassilevski,et al.  Preconditioning of Indefinite and Almost Singular Finite Element Elliptic Equations , 1998, SIAM J. Sci. Comput..

[31]  Yin Zhang,et al.  Solving large-scale linear programs by interior-point methods under the Matlab ∗ Environment † , 1998 .

[32]  G. W. Stewart,et al.  The QLP Approximation to the Singular Value Decomposition , 1999, SIAM J. Sci. Comput..

[33]  M. Gutknecht,et al.  Residual Smoothing Techniques: Do They Improve the Limiting Accuracy of Iterative Solvers? , 1999 .

[34]  Gene H. Golub,et al.  A Note on Preconditioning for Indefinite Linear Systems , 1999, SIAM J. Sci. Comput..

[35]  Misha Elena Kilmer,et al.  Iterative Regularization and MINRES , 1999, SIAM J. Matrix Anal. Appl..

[36]  William W. Hager,et al.  Iterative Methods for Nearly Singular Linear Systems , 2000, SIAM J. Sci. Comput..

[37]  J. Demmel,et al.  Balancing sparse matrices for computing eigenvalues , 2000 .

[38]  Gerard L. G. Sleijpen,et al.  Differences in the Effects of Rounding Errors in Krylov Solvers for Symmetric Indefinite Linear Systems , 2000, SIAM J. Matrix Anal. Appl..

[39]  Stephen A. Vavasis,et al.  Accurate Solution of Weighted Least Squares by Iterative Methods , 2000, SIAM J. Matrix Anal. Appl..

[40]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[41]  M. Benzi.  Preconditioning techniques for large linear systems: a survey , 2002 .

[42]  Valeria Simoncini,et al.  Krylov Subspace Methods for Saddle Point Problems with Indefinite Preconditioning , 2002, SIAM J. Matrix Anal. Appl..

[43]  Z. Bai,et al.  Restrictively preconditioned conjugate gradient methods for systems of linear equations , 2003 .

[44]  K. Toh,et al.  Block preconditioners for symmetric indefinite linear systems , 2004 .

[45]  Gene H. Golub,et al.  Scaling by Binormalization , 2004, Numerical Algorithms.

[46]  M. Arioli,et al.  Least-squares problems, normal equations, and stopping criteria for the conjugate gradient method , 2008 .

[47]  Kaustuv.  IPSOL: An interior point solver for nonconvex optimization problems , 2009 .

[48]  Xiao-Wen Chang,et al.  Stopping Criteria for the Iterative Solution of Linear Least Squares Problems , 2009, SIAM J. Matrix Anal. Appl..

[49]  D. Bernstein.  Matrix Mathematics: Theory, Facts, and Formulas , 2009 .

[50]  A. Bradley.  Algorithms for the Equilibration of Matrices and Their Application to Limited-Memory Quasi-Newton Methods , 2010 .

[51]  Pavel Jiránek,et al.  Estimating the Backward Error in LSQR , 2010, SIAM J. Matrix Anal. Appl..

[52]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[53]  G. Meurant.  Least Squares Problems , 2018, Explorations in Numerical Analysis.