Estimating the Backward Error in LSQR

We propose practical stopping criteria for the iterative solution of sparse linear least squares (LS) problems. Although we focus our discussion on the algorithm LSQR of Paige and Saunders, the ideas discussed here may also be applicable to other algorithms. We review why the 2-norm of the projection of the residual vector onto the range of $A$ is a useful measure of convergence, and we show how this projection can be estimated efficiently at every iteration of LSQR. We also give practical and cheaply computable estimates of the backward error for the LS problem.

[1]  Ming Gu,et al.  Backward Perturbation Bounds for Linear Least Squares Problems , 1999, SIAM J. Matrix Anal. Appl..

[2]  Gene H. Golub,et al.  Matrix computations , 1983 .

[3]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[4]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[5]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[6]  Michael A. Saunders,et al.  Algorithm 583: LSQR: Sparse Linear Equations and Least Squares Problems , 1982, TOMS.

[7]  Xiao-Wen Chang,et al.  Stopping Criteria for the Iterative Solution of Linear Least Squares Problems , 2009, SIAM J. Matrix Anal. Appl..

[8]  R. Karlson,et al.  Estimation of optimal backward perturbation bounds for the linear least squares problem , 1997 .

[9]  Å. Björck,et al.  Stability of Conjugate Gradient and Lanczos Methods for Linear Least Squares Problems , 1998, SIAM J. Matrix Anal. Appl..

[10]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[11]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[12]  Ji-Guang Sun,et al.  Optimal backward perturbation bounds for the linear least squares problem , 1995, Numer. Linear Algebra Appl..

[13]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[14]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[15]  C. Paige.  Bidiagonalization of Matrices and Solution of Linear Equations , 1974 .

[16]  M. Arioli,et al.  Least-squares problems, normal equations, and stopping criteria for the conjugate gradient method , 2008 .

[17]  Z. Strakos,et al.  Error Estimation in Preconditioned Conjugate Gradients , 2005 .

[18]  Richard F. Barrett,et al.  Matrix Market: a web resource for test matrix collections , 1996, Quality of Numerical Software.

[19]  Gene H. Golub,et al.  Calculating the singular values and pseudo-inverse of a matrix , 2007, Milestones in Matrix Computation.

[20]  Z. Strakos,et al.  On error estimation in the conjugate gradient method and why it works in finite precision computations. , 2002 .

[21]  C. Lanczos.  An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .