On an iterative method for solving the least squares problem of rank-deficient systems

In this paper, we first show an iterative method for finding the least squares (LS) solution to the inconsistent system Ax=b, where A is an m×n matrix of rank r. The method is an iteration scheme for consistent system of linear equations M=bˆ which is an augmented system associated with Ax=b. It denotes that under some conditions, the sequence, 0, 1, 2, … , converges to the LS solution of the system Ax=b for every initial vector 0, where (M+γ E)i=γ Ei−1+bˆ, for i=1, 2, … . In our numerical test, we propose to find E without using decomposition methods. The improved timings are shown with matrices of substantial size.

[1]  Apostolos Hadjidimos,et al.  Accelerated overrelaxation method , 1978 .

[2]  C. Paige.  Bidiagonalization of Matrices and Solution of Linear Equations , 1974 .

[3]  Dianne P. O'Leary,et al.  Complete stagnation of gmres , 2003 .

[4]  James Demmel,et al.  Extra-Precise Iterative Refinement for Overdetermined Least Squares Problems , 2009, TOMS.

[5]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[6]  Iain S. Duff,et al.  A Class of Incomplete Orthogonal Factorization Methods. I: Methods and Theories , 1999 .

[7]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[8]  Successive Overrelaxation Methods for Solving the Rank Deficient Linear Least Squares Problem , 2001 .

[9]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[10]  Hongjiong Tian.  Accelerate overrelaxation methods for rank deficient linear systems , 2003, Appl. Math. Comput..

[11]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[12]  G. Stewart.  Introduction to matrix computations , 1973 .

[13]  Iain S. Duff,et al.  A Class of Incomplete Orthogonal Factorization Methods. II: Implementation and Results , 2005 .

[14]  Valerie A. Miller,et al.  Successive overrelaxation methods for solving the rank deficient linear squares problem , 1987 .

[15]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[16]  Robert J. Plemmons,et al.  Nonnegative Matrices in the Mathematical Sciences , 1979, Classics in Applied Mathematics.