Convergence of Inner-Iteration GMRES Methods for Rank-Deficient Least Squares Problems

We develop a general convergence theory for the generalized minimal residual method preconditioned by inner iterations for solving least squares problems. The inner iterations are performed by stationary iterative methods. We also present theoretical justifications for using specific inner iterations such as the Jacobi and SOR-type methods. The theory improves previous work [K. Morikuni and K. Hayami, SIAM J. Matrix Anal. Appl., 34 (2013), pp. 1--22], particularly in the rank-deficient case. We also characterize the spectrum of the preconditioned coefficient matrix by the spectral radius of the iteration matrix for the inner iterations and give a convergence bound for the proposed methods. Finally, numerical experiments show that the proposed methods are more robust and efficient compared to previous methods for some rank-deficient problems.

[1]  H. Keller.  On the Solution of Singular and Semidefinite Linear Systems by Iteration , 1965 .

[2]  C. Radhakrishna Rao,et al.  Generalized inverse of linear transformations: a geometric approach , 1985 .

[3]  R. Oldenburger,et al.  Infinite powers of matrices and characteristic roots , 1940 .

[4]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[5]  H. Walker,et al.  GMRES On (Nearly) Singular Systems , 1997, SIAM J. Matrix Anal. Appl..

[6]  C. D. Meyer,et al.  Convergent Powers of a Matrix With Applications to Iterative Methods for Singular Linear Systems , 1975 .

[7]  Anne Greenbaum,et al.  Any Nonincreasing Convergence Curve is Possible for GMRES , 1996, SIAM J. Matrix Anal. Appl..

[8]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[9]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[10]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[11]  Zhong-Zhi Bai,et al.  Sharp error bounds of some Krylov subspace methods for non-Hermitian linear systems , 2000, Appl. Math. Comput..

[12]  Jun-Feng Yin,et al.  GMRES Methods for Least Squares Problems , 2010, SIAM J. Matrix Anal. Appl..

[13]  E. J. Craig.  The N‐Step Iteration Procedures , 1955 .

[14]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[15]  K. Tanabe.  Characterization of linear stationary iterative processes for solving a singular system of linear equations , 1974 .

[16]  Å. Björck,et al.  Accelerated projection methods for computing pseudoinverse solutions of systems of linear equations , 1979 .

[17]  Keiichi Morikuni,et al.  Inner-Iteration Krylov Subspace Methods for Least Squares Problems , 2013, SIAM J. Matrix Anal. Appl..

[18]  Achiya Dax,et al.  The Convergence of Linear Stationary Iterative Processes for Solving Singular Unstructured Systems of Linear Equations , 1990, SIAM Rev..

[19]  Laurent Smoch,et al.  Spectral behaviour of GMRES applied to singular systems , 2007, Adv. Comput. Math..

[20]  N. S. Barnett,et al.  Private communication , 1969 .

[21]  Masaaki Sugihara,et al.  A geometric view of Krylov subspace methods on singular systems , 2011, Numer. Linear Algebra Appl..

[22]  M. Arioli,et al.  Krylov sequences of maximal length and convergence of GMRES , 1997 .

[23]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .

[24]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[25]  Kurt Hensel.  Über Potenzreihen von Matrizen. , 1926 .

[26]  Jun-Feng Yin,et al.  Greville’s method for preconditioning least squares problems , 2011, Adv. Comput. Math..

[27]  ZhangShaoLiang,et al.  Orthomin(k) Method for Linear Least Squares Problem , 1991 .

[28]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[29]  Gérard Meurant,et al.  Prescribing the behavior of early terminating GMRES and Arnoldi iterations , 2013, Numerical Algorithms.

[30]  Michael A. Saunders,et al.  LSMR: An Iterative Algorithm for Sparse Least-Squares Problems , 2010, SIAM J. Sci. Comput..