Implicitly Restarted GMRES and Arnoldi Methods for Nonsymmetric Systems of Equations

The generalized minimum residual method (GMRES) is well known for solving large nonsymmetric systems of linear equations. It generally uses restarting, which slows the convergence. However, some information can be retained at the time of the restart and used in the next cycle. We present algorithms that use implicit restarting in order to retain this information. Approximate eigenvectors determined from the previous subspace are included in the new subspace. This deflates the smallest eigenvalues and thus improves the convergence. The subspace that contains the approximate eigenvectors is itself a Krylov subspace, but not with the usual starting vector. The implicitly restarted FOM algorithm includes standard Ritz vectors in the subspace. The eigenvalue portion of its calculations is equivalent to Sorensen's IRA algorithm. The implicitly restarted GMRES algorithm uses harmonic Ritz vectors. This algorithm also gives a new approach to computing interior eigenvalues.

[1]  T. Manteuffel,et al.  A taxonomy for conjugate gradient methods , 1990 .

[2]  G. Golub,et al.  Gmres: a Generalized Minimum Residual Algorithm for Solving , 2022 .

[3]  Ronald B. Morgan,et al.  A Restarted GMRES Method Augmented with Eigenvectors , 1995, SIAM J. Matrix Anal. Appl..

[4]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[5]  D. Sorensen.  Numerical methods for large eigenvalue problems , 2002, Acta Numerica.

[6]  Danny C. Sorensen,et al.  Deflation Techniques for an Implicitly Restarted Arnoldi Iteration , 1996, SIAM J. Matrix Anal. Appl..

[7]  Yousef Saad,et al.  A Flexible Inner-Outer Preconditioned GMRES Algorithm , 1993, SIAM J. Sci. Comput..

[8]  Roland W. Freund,et al.  An Implementation of the Look-Ahead Lanczos Algorithm for Non-Hermitian Matrices , 1993, SIAM J. Sci. Comput..

[9]  P. Sonneveld.  CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems , 1989 .

[10]  H. V. D. Vorst,et al.  The superlinear convergence behaviour of GMRES , 1993 .

[11]  Dennis C. Smolarski,et al.  Implementation of an adaptive algorithm for Richardson's method , 1991 .

[12]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[13]  Danny C. Sorensen,et al.  Implicit Application of Polynomial Filters in a k-Step Arnoldi Method , 1992, SIAM J. Matrix Anal. Appl..

[14]  R. Freund,et al.  QMR: a quasi-minimal residual method for non-Hermitian linear systems , 1991 .

[15]  S. A. Kharchenko,et al.  Eigenvalue translation based preconditioners for the GMRES(k) method , 1995, Numer. Linear Algebra Appl..

[16]  Lloyd N. Trefethen,et al.  GMRES/CR and Arnoldi/Lanczos as Matrix Approximation Problems , 2018, SIAM J. Sci. Comput..

[17]  Gerard L. G. Sleijpen,et al.  A Jacobi-Davidson Iteration Method for Linear Eigenvalue Problems , 1996, SIAM Rev..

[18]  H. A. V. D. Vorsty.  University Utrecht a Generalized Jacobi-davidson Iteration Method for Linear Eigenvalue Problems a Generalized Jacobi-davidson Iteration Method for Linear Eigenvalue Problems , 1994 .

[19]  W. Arnoldi.  The principle of minimized iterations in the solution of the matrix eigenvalue problem , 1951 .

[20]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[21]  Anne Greenbaum,et al.  Any Nonincreasing Convergence Curve is Possible for GMRES , 1996, SIAM J. Matrix Anal. Appl..

[22]  R. Nicolaides.  Deflation of conjugate gradients with applications to boundary value problems , 1987 .

[23]  Gene H. Golub,et al.  Matrix computations , 1983 .

[24]  K. Burrage,et al.  Restarted GMRES preconditioned by deflation , 1996 .

[25]  C. Lanczos.  Solution of Systems of Linear Equations by Minimized Iterations1 , 1952 .

[26]  R. Morgan.  Computing Interior Eigenvalues of Large Matrices , 1991 .

[27]  E. Sturler,et al.  Truncation Strategies for Optimal Krylov Subspace Methods , 1999 .

[28]  Yousef Saad,et al.  Deflated and Augmented Krylov Subspace Techniques , 1997, Numer. Linear Algebra Appl..

[29]  Y. Saad.  Analysis of Augmented Krylov Subspace Methods , 1997, SIAM J. Matrix Anal. Appl..

[30]  Gene H. Golub,et al.  Some History of the Conjugate Gradient and Lanczos Algorithms: 1948-1976 , 1989, SIAM Rev..

[31]  Thomas A. Manteuffel,et al.  On hybrid iterative methods for nonsymmetric systems of linear equations , 1996 .

[32]  Ronald B. Morgan,et al.  On restarting the Arnoldi method for large nonsymmetric eigenvalue problems , 1996, Math. Comput..

[33]  Henk A. van der Vorst,et al.  Approximate solutions and eigenvalue bounds from Krylov subspaces , 1995, Numer. Linear Algebra Appl..

[34]  Gerard L. G. Sleijpen,et al.  A Jacobi-Davidson Iteration Method for Linear Eigenvalue Problems , 1996, SIAM J. Matrix Anal. Appl..

[35]  L. Knizhnerman,et al.  Two polynomial methods of calculating functions of symmetric matrices , 1991 .

[36]  R. Lehoucq.  Analysis and implementation of an implicitly restarted Arnoldi iteration , 1996 .

[37]  Roland W. Freund,et al.  A Transpose-Free Quasi-Minimal Residual Algorithm for Non-Hermitian Linear Systems , 1993, SIAM J. Sci. Comput..

[38]  Kesheng Wu,et al.  Dynamic Thick Restarting of the Davidson, and the Implicitly Restarted Arnoldi Methods , 1998, SIAM J. Sci. Comput..

[39]  C. Lanczos.  An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[40]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[41]  Y. Saad.  Krylov subspace methods for solving large unsymmetric linear systems , 1981 .

[42]  G. Golub,et al.  Iterative solution of linear systems , 1991, Acta Numerica.

[43]  Wayne Joubert,et al.  A Robust GMRES-Based Adaptive Polynomial Preconditioning Algorithm for Nonsymmetric Linear Systems , 1994, SIAM J. Sci. Comput..

[44]  Y. Saad.  Variations on Arnoldi's method for computing eigenelements of large unsymmetric matrices , 1980 .

[45]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[46]  R. Freund.  Quasi-kernel polynomials and their use in non-Hermitian matrix iterations , 1992 .

[47]  Cornelis Vuik,et al.  GMRESR: a family of nested GMRES methods , 1994, Numer. Linear Algebra Appl..

[48]  E. Desturler,et al.  Nested Krylov methods and preserving the orthogonality , 1993 .

[49]  Min Zeng,et al.  Harmonic projection methods for large non-symmetric eigenvalue problems , 1998, Numer. Linear Algebra Appl..

[50]  Iain S. Duff,et al.  Sparse matrix test problems , 1982 .

[51]  Zhishun A. Liu,et al.  A Look Ahead Lanczos Algorithm for Unsymmetric Matrices , 1985 .

[52]  Lloyd N. Trefethen,et al.  A Hybrid GMRES Algorithm for Nonsymmetric Linear Systems , 1992, SIAM J. Matrix Anal. Appl..

[53]  Thomas A. Manteuffel,et al.  On the roots of the orthogonal polynomials and residual polynomials associated with a conjugate gradient method , 1994, Numer. Linear Algebra Appl..

[54]  Gene H. Golub,et al.  Adaptively Preconditioned GMRES Algorithms , 1998, SIAM J. Sci. Comput..