Recycling Krylov Subspaces for Sequences of Linear Systems

Many problems in science and engineering require the solution of a long sequence of slowly changing linear systems. We propose and analyze two methods that significantly reduce the total number of matrix-vector products required to solve all systems. We consider the general case where both the matrix and right-hand side change, and we make no assumptions regarding the change in the right-hand sides. Furthermore, we consider general nonsingular matrices, and we do not assume that all matrices are pairwise close or that the sequence of matrices converges to a particular matrix. Our methods work well under these general assumptions, and hence form a significant advancement with respect to related work in this area. We can reduce the cost of solving subsequent systems in the sequence by recycling selected subspaces generated for previous systems. We consider two approaches that allow for the continuous improvement of the recycled subspace at low cost. We consider both Hermitian and non-Hermitian problems, and we analyze our algorithms both theoretically and numerically to illustrate the effects of subspace recycling. We also demonstrate the effectiveness of our algorithms for a range of applications from computational mechanics, materials science, and computational physics.

[1]  Paul Fischer,et al.  PROJECTION TECHNIQUES FOR ITERATIVE SOLUTION OF Ax = b WITH SUCCESSIVE RIGHT-HAND SIDES , 1993 .

[2]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[3]  A. V. Smirnov,et al.  Accuracy and limitations of localized Green’s function methods for materials science applications , 2001 .

[4]  E. Sturler,et al.  Truncation Strategies for Optimal Krylov Subspace Methods , 1999 .

[5]  Ronald B. Morgan,et al.  GMRES WITH DEFLATED , 2008 .

[6]  Oliver G. Ernst,et al.  Analysis of acceleration strategies for restarted minimal residual methods , 2000 .

[7]  G. W. Stewart,et al.  Matrix algorithms , 1998 .

[8]  H. V. D. Vorst,et al.  The superlinear convergence behaviour of GMRES , 1993 .

[9]  Charbel Farhat,et al.  Implicit parallel processing in structural mechanics , 1994 .

[10]  Valeria Simoncini,et al.  On the Superlinear Convergence of Exact and Inexact Krylov Subspace Methods , 2003 .

[11]  K. Burrage,et al.  Restarted GMRES preconditioned by deflation , 1996 .

[12]  Gene H. Golub,et al.  Adaptively Preconditioned GMRES Algorithms , 1998, SIAM J. Sci. Comput..

[13]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[14]  Misha Elena Kilmer,et al.  Recycling Subspace Information for Diffuse Optical Tomography , 2005, SIAM J. Sci. Comput..

[15]  Ronald B. Morgan,et al.  A Restarted GMRES Method Augmented with Eigenvectors , 1995, SIAM J. Matrix Anal. Appl..

[16]  Jack Dongarra,et al.  Numerical Linear Algebra for High-Performance Computers , 1998 .

[17]  H. V. D. Vorst,et al.  Numerical methods for the QCDd overlap operator. I. Sign-function and error bounds , 2002, hep-lat/0202025.

[18]  Michael Creutz,et al.  Quarks, Gluons and Lattices , 1984 .

[19]  J. Korringa,et al.  On the calculation of the energy of a Bloch wave in a metal , 1947 .

[20]  S. Eisenstat,et al.  Variational Iterative Methods for Nonsymmetric Systems of Linear Equations , 1983 .

[21]  Kesheng Wu,et al.  Dynamic Thick Restarting of the Davidson, and the Implicitly Restarted Arnoldi Methods , 1998, SIAM J. Sci. Comput..

[22]  E. Sturler,et al.  Nested Krylov methods based on GCR , 1996 .

[23]  Michael K. Ng,et al.  Galerkin Projection Methods for Solving Multiple Linear Systems , 1999, SIAM J. Sci. Comput..

[24]  Johnson,et al.  Total-energy and pressure calculations for random substitutional alloys. , 1990, Physical review. B, Condensed matter.

[25]  Christian Rey,et al.  A Rayleigh–Ritz preconditioner for the iterative solution to large scale nonlinear problems , 1998, Numerical Algorithms.

[26]  Richard F. Barrett,et al.  Matrix Market: a web resource for test matrix collections , 1996, Quality of Numerical Software.

[27]  Ronald B. Morgan,et al.  Implicitly Restarted GMRES and Arnoldi Methods for Nonsymmetric Systems of Equations , 2000, SIAM J. Matrix Anal. Appl..

[28]  W. Kohn,et al.  Solution of the Schrödinger Equation in Periodic Lattices with an Application to Metallic Lithium , 1954 .

[29]  Arne S. Gullerud,et al.  MPI-based implementation of a PCG solver using an EBE architecture and preconditioner for implicit, 3-D finite element analysis , 2001 .

[30]  Weinberger,et al.  Theory and convergence properties of the screened Korringa-Kohn-Rostoker method. , 1995, Physical review. B, Condensed matter.

[31]  Valeria Simoncini,et al.  On the Occurrence of Superlinear Convergence of Exact and Inexact Krylov Subspace Methods , 2005, SIAM Rev..

[32]  Gene H. Golub,et al.  Matrix computations , 1983 .

[33]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[34]  Frédéric Guyomarc'h,et al.  A Deflated Version of the Conjugate Gradient Algorithm , 1999, SIAM J. Sci. Comput..

[35]  Kyle A. Gallivan,et al.  A new family of block methods , 1999 .

[36]  Eric de Sturler,et al.  Analysis of Krylov subspace recycling for sequences of linear systems. , 2005 .

[37]  G. Stewart.  Matrix Algorithms, Volume II: Eigensystems , 2001 .

[38]  D. O’Leary.  The block conjugate gradient algorithm and related methods , 1980 .

[39]  Eric de Sturler,et al.  The iterative solution of a sequence of linear systems arising from nonlinear finite element analysis , 2005 .

[40]  John Rossi,et al.  Convergence of Restarted Krylov Subspaces to Invariant Subspaces , 2004, SIAM J. Matrix Anal. Appl..

[41]  B. Vital.  Etude de quelques methodes de resolution de problemes lineaires de grande taille sur multiprocesseur , 1990 .

[42]  Efstratios Gallopoulos,et al.  An Iterative Method for Nonsymmetric Systems with Multiple Right-Hand Sides , 1995, SIAM J. Sci. Comput..

[43]  Y. Saad.  Analysis of Augmented Krylov Subspace Methods , 1997, SIAM J. Matrix Anal. Appl..

[44]  Christian Rey,et al.  Iterative accelerating algorithms with Krylov subspaces for the solution to large-scale nonlinear problems , 2004, Numerical Algorithms.

[45]  Kesheng Wu,et al.  Thick-Restart Lanczos Method for Large Symmetric Eigenvalue Problems , 2000, SIAM J. Matrix Anal. Appl..