On the Convergence of GMRES with Invariant-Subspace Deflation

We consider the solution of large and sparse linear systems of equations by GMRES. Due to the appearance of unfavorable eigenvalues in the spectrum of the coefficient matrix, the convergence of GMRES may hamper. To overcome this, a deflated variant of GMRES can be used, which treats those unfavorable eigenvalues effectively. In the literature, several deflated GMRES variants are applied successfully to various problems, while a theoretical justification is often lacking. In contrast to deflated CG, the convergence of deflated GMRES seems to be harder to analyze and to understand. This paper presents some new theoretical insights into deflated GMRES based on A-invariant deflation subspaces. Fundamental results regarding the convergence of deflated GMRES are proved in order to show the effectiveness and robustness of this method. Numerical experiments are provided to illustrate the theoretical results and to show some further properties of deflated GMRES. Consequently, practical variants of deflated GMRES from the literature can be better understood based on the results presented in this paper.

[1]  Cornelis Vuik,et al.  A Comparison of Deflation and Coarse Grid Correction Applied to Porous Media Flow , 2004, SIAM J. Numer. Anal..

[2]  J. Meijerink,et al.  An Efficient Preconditioned CG Method for the Solution of a Class of Layered Problems with Extreme Contrasts in the Coefficients , 1999 .

[3]  Cornelis Vuik,et al.  Comparison of Two-Level Preconditioners Derived from Deflation, Domain Decomposition and Multigrid Methods , 2009, J. Sci. Comput..

[4]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[5]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .

[6]  H. V. D. Vorst,et al.  The superlinear convergence behaviour of GMRES , 1993 .

[7]  Eric de Sturler,et al.  Recycling Krylov Subspaces for Sequences of Linear Systems , 2006, SIAM J. Sci. Comput..

[8]  Tony F. Chan,et al.  ML(k)BiCGSTAB: A BiCGSTAB Variant Based on Multiple Lanczos Starting Vectors , 1999, SIAM J. Sci. Comput..

[9]  K. Burrage,et al.  Restarted GMRES preconditioned by deflation , 1996 .

[10]  Gene H. Golub,et al.  Adaptively Preconditioned GMRES Algorithms , 1998, SIAM J. Sci. Comput..

[11]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[12]  G. Meinardus,et al.  Über eine Verallgemeinerung einer Ungleichung von L. V. Kantorowitsch , 1963 .

[13]  E. Sturler,et al.  Truncation Strategies for Optimal Krylov Subspace Methods , 1999 .

[14]  Cornelis Vuik,et al.  A comparison of abstract versions of deflation, balancing and additive coarse grid correction preconditioners , 2008, Numer. Linear Algebra Appl..

[15]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[16]  Z. Dostál.  Conjugate gradient method with preconditioning by projector , 1988 .

[17]  A. Jennings.  Influence of the Eigenvalue Spectrum on the Convergence Rate of the Conjugate Gradient Method , 1977 .

[18]  Yousef Saad,et al.  Deflated and Augmented Krylov Subspace Techniques , 1997, Numer. Linear Algebra Appl..

[19]  S. Eisenstat,et al.  Variational Iterative Methods for Nonsymmetric Systems of Linear Equations , 1983 .

[20]  R. Nicolaides.  Deflation of conjugate gradients with applications to boundary value problems , 1987 .

[21]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[22]  Frédéric Guyomarc'h,et al.  A Deflated Version of the Conjugate Gradient Algorithm , 1999, SIAM J. Sci. Comput..

[23]  S. A. Kharchenko,et al.  Eigenvalue translation based preconditioners for the GMRES(k) method , 1995, Numer. Linear Algebra Appl..

[24]  T. Manteuffel.  The Tchebychev iteration for nonsymmetric linear systems , 1977 .

[25]  Ronald B. Morgan,et al.  A Restarted GMRES Method Augmented with Eigenvectors , 1995, SIAM J. Matrix Anal. Appl..

[26]  R. Morgan,et al.  Deflated GMRES for systems with multiple shifts and multiple right-hand sides☆ , 2007, 0707.0502.

[27]  Cornelis Vuik,et al.  Fast and robust solvers for pressure-correction in bubbly flow problems , 2008, J. Comput. Phys..

[28]  Y. Saad.  Analysis of Augmented Krylov Subspace Methods , 1997, SIAM J. Matrix Anal. Appl..

[29]  H. V. D. Vorst,et al.  The rate of convergence of Conjugate Gradients , 1986 .

[30]  Cornelis Vuik,et al.  On the Construction of Deflation-Based Preconditioners , 2001, SIAM J. Sci. Comput..

[31]  W. Arnoldi.  The principle of minimized iterations in the solution of the matrix eigenvalue problem , 1951 .

[32]  Alan B. Williams,et al.  A Deflation Technique for Linear Systems of Equations , 1998, SIAM J. Sci. Comput..

[33]  S. Kaniel.  Estimates for Some Computational Techniques - in Linear Algebra , 1966 .

[34]  Anne Greenbaum,et al.  Any Nonincreasing Convergence Curve is Possible for GMRES , 1996, SIAM J. Matrix Anal. Appl..

[35]  Misha Elena Kilmer,et al.  Recycling Subspace Information for Diffuse Optical Tomography , 2005, SIAM J. Sci. Comput..

[36]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[37]  Martin B. van Gijzen,et al.  IDR(s): A Family of Simple and Fast Algorithms for Solving Large Nonsymmetric Systems of Linear Equations , 2008, SIAM J. Sci. Comput..