A Comparative Study of Iterative Solvers Exploiting Spectral Information for SPD Systems

When solving the symmetric positive definite (SPD) linear system ${\bf A} {\bf x}^\star = {\bf b}$ with the conjugate gradient method, the smallest eigenvalues in the matrix ${\bf A}$ often slow down the convergence. Consequently if the smallest eigenvalues in ${\bf A}$ could somehow be "removed," the convergence may be improved. This observation is of importance even when a preconditioner is used, and some extra techniques might be investigated to further improve the convergence rate of the conjugate gradient on the given preconditioned system. Several techniques have been proposed in the literature that consist of either updating the preconditioner or enforcing conjugate gradient to work in the orthogonal complement of an invariant subspace associated with the smallest eigenvalues. The goal of this work is to compare several of these techniques in terms of numerical efficiency. Among various possibilities, we exploit the Partial Spectral Factorization algorithm presented in [M. Arioli and D. Ruiz, Technical Report RAL-TR-2002-021, Rutherford Appleton Laboratory, Atlas Center, Didcot, Oxfordshire, England, 2002] to compute an orthonormal basis of a near-invariant subspace of ${\bf A}$ associated with the smallest eigenvalues. This eigeninformation is used in combination with different solution techniques. In particular we consider the deflated version of conjugate gradient. As representative of techniques exploiting the spectral information to update the preconditioner we consider also the approaches that attempt to shift the smallest eigenvalues close to one where most of the eigenvalues of the preconditioned matrix should be located. Finally, we consider an algebraic two-grid scheme inspired by ideas from the multigrid philosophy. In this paper, we describe these various variants and we compare their numerical behavior on a set of model problems from Matrix Market or arising from the discretization via the finite element technique of some two-dimensional (2D) heterogeneous diffusion PDE problems. We discuss their numerical efficiency, computational complexity, and sensitivity to the accuracy of the eigencalculation.

[1]  Luc Giraud,et al.  Krylov and Polynomial Iterative Solvers Combined with Partial Spectral Factorization for SPD Linear Systems , 2004, VECPAR.

[2]  Z. Dostál.  Conjugate gradient method with preconditioning by projector , 1988 .

[3]  L. Yu. Kolotilina,et al.  Twofold deflation preconditioning of linear algebraic systems. I. Theory , 1998 .

[4]  B. Parlett.  A new look at the Lanczos algorithm for solving symmetric systems of linear equations , 1980 .

[5]  Cornelis Vuik,et al.  The influence of deflation vectors at interfaces on the deflated conjugate gradient method , 2001 .

[6]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[7]  Frédéric Guyomarc'h,et al.  A Deflated Version of the Conjugate Gradient Algorithm , 1999, SIAM J. Sci. Comput..

[8]  Y. Saad,et al.  On the Lánczos method for solving symmetric linear systems with several right-hand sides , 1987 .

[9]  Ronald B. Morgan,et al.  Implicitly Restarted GMRES and Arnoldi Methods for Nonsymmetric Systems of Equations , 2000, SIAM J. Matrix Anal. Appl..

[10]  Tony F. Chan,et al.  Analysis of Projection Methods for Solving Linear Systems with Multiple Right-Hand Sides , 1997, SIAM J. Sci. Comput..

[11]  R. Nicolaides.  Deflation of conjugate gradients with applications to boundary value problems , 1987 .

[12]  Gene H. Golub,et al.  The block Lanczos method for computing eigenvalues , 2007, Milestones in Matrix Computation.

[13]  Chao Yang,et al.  ARPACK users' guide - solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods , 1998, Software, environments, tools.

[14]  Ronald B. Morgan,et al.  A Restarted GMRES Method Augmented with Eigenvectors , 1995, SIAM J. Matrix Anal. Appl..

[15]  David M. Young,et al.  Applied Iterative Methods , 2004 .

[16]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[17]  G. Golub,et al.  Estimates of Eigenvalues for Iterative Methods , 1989 .

[18]  Frédéric Guyomarc'h,et al.  An Augmented Conjugate Gradient Method for Solving Consecutive Symmetric Positive Definite Linear Systems , 2000, SIAM J. Matrix Anal. Appl..

[19]  E. F. Kaasschieter,et al.  Preconditioned conjugate gradients for solving singular systems , 1988 .

[20]  Gene H. Golub,et al.  Adaptively Preconditioned GMRES Algorithms , 1998, SIAM J. Sci. Comput..

[21]  Y. Saad.  On the Rates of Convergence of the Lanczos and the Block-Lanczos Methods , 1980 .

[22]  John G. Lewis,et al.  Sparse matrix test problems , 1982, SGNM.

[23]  Kay Hameyer,et al.  A deflated iterative solver for magnetostatic finite element models with large differences in permeability , 2001 .

[24]  Gene H. Golub,et al.  A generalized conjugate gradient method for the numerical solution of elliptic partial differential equations , 2007, Milestones in Matrix Computation.

[25]  Bruno Carpentieri,et al.  A Class of Spectral Two-Level Preconditioners , 2003, SIAM J. Sci. Comput..

[26]  D. O’Leary.  A generalized conjugate gradient algorithm for solving a class of quadratic programming problems , 1977 .

[27]  H. V. D. Vorst,et al.  An iterative solution method for solving f ( A ) x = b , using Krylov subspace information obtained for the symmetric positive definite matrix A , 1987 .

[28]  Richard R. Underwood.  An iterative block Lanczos method for the solution of large sparse symmetric eigenproblems , 1975 .

[29]  M Arioli,et al.  A Chebyshev-based two-stage iterative method as an alternative to the direct solution of linear systems , 2002 .

[30]  H. V. D. Vorst,et al.  The rate of convergence of Conjugate Gradients , 1986 .

[31]  C. Paige.  Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem , 1980 .

[32]  Cornelis Vuik,et al.  A Comparison of Deflation and Coarse Grid Correction Applied to Porous Media Flow , 2004, SIAM J. Numer. Anal..

[33]  D. Ruiz,et al.  Adaptive preconditioners for nonlinear systems of equations , 2006 .