Efficient and reliable iterative methods for linear systems

Abstract The approximate solutions in standard iteration methods for linear systems Ax=b, with A an n by n nonsingular matrix, form a subspace. In this subspace, one may try to construct better approximations for the solution x. This is the idea behind Krylov subspace methods. It has led to very powerful and efficient methods such as conjugate gradients, GMRES, and Bi-CGSTAB. We will give an overview of these methods and we will discuss some relevant properties from the user's perspective view. The convergence of Krylov subspace methods depends strongly on the eigenvalue distribution of A, and on the angles between eigenvectors of A. Preconditioning is a popular technique to obtain a better behaved linear system. We will briefly discuss some modern developments in preconditioning, in particular parallel preconditioners will be highlighted: reordering techniques for incomplete decompositions, domain decomposition approaches, and sparsified Schur complements.

[1]  Henk A. van der Vorst,et al.  Spectral analysis of parallel incomplete factorizations with implicit pseudo‐overlap , 2002, Numer. Linear Algebra Appl..

[2]  W. Arnoldi The principle of minimized iterations in the solution of the matrix eigenvalue problem , 1951 .

[3]  D. Kershaw The incomplete Cholesky—conjugate gradient method for the iterative solution of systems of linear equations , 1978 .

[4]  Jack Dongarra,et al.  Numerical Linear Algebra for High-Performance Computers , 1998 .

[5]  Henk A. van der Vorst,et al.  Large tridiagonal and block tridiagonal linear systems on vector and parallel computers , 1987, Parallel Comput..

[6]  J. Reid The Use of Conjugate Gradients for Systems of Linear Equations Possessing “Property A” , 1972 .

[7]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[8]  O. Axelsson Iterative solution methods , 1995 .

[9]  Yvan Notay,et al.  DRIC: A dynamic version of the RIC method , 1994, Numer. Linear Algebra Appl..

[10]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[11]  Henk A. van der Vorst,et al.  A generalized domain decomposition paradigm for parallel incomplete LU factorization preconditionings , 2001, Future Gener. Comput. Syst..

[12]  J. Meijerink,et al.  Guidelines for the usage of incomplete decompositions in solving sets of linear equations as they occur in practical problems , 1981 .

[13]  Auke van der Ploeg,et al.  Preconditioning for spare matrices with applications , 1994 .

[14]  P. Sonneveld CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems , 1989 .

[15]  Fred Wubs,et al.  Nested grids ILU-decomposition (NGILU) , 1996 .

[16]  Anne Greenbaum,et al.  Iterative methods for solving linear systems , 1997, Frontiers in applied mathematics.

[17]  Gérard Meurant,et al.  NUMERICAL EXPERIMENTS FOR THE PRECONDITIONED CONJUGATE GRADIENT METHOD ON THE CRAY X-MP/2 , 1984 .

[18]  Henk A. van der Vorst,et al.  Developments and trends in the parallel solution of linear systems , 1999, Parallel Comput..

[19]  Gundolf Haase Parallel Incomplete Cholesky Preconditioners Based on the Non-Overlapping Data Distribution , 1998, Parallel Comput..

[20]  Wei-Pai Tang,et al.  Ordering Methods for Preconditioned Conjugate Gradient Methods Applied to Unstructured Grid Problems , 1992, SIAM J. Matrix Anal. Appl..

[21]  E. De Sturler Iterative methods on distributed memory computers , 1994 .

[22]  H. V. D. Vorst,et al.  Iterative solution methods for certain sparse linear systems with a non-symmetric matrix arising from PDE-problems☆ , 1981 .

[23]  Richard Barrett,et al.  Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods , 1994, Other Titles in Applied Mathematics.

[24]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[25]  H. V. D. Vorst,et al.  A parallel linear system solver for circuit simulation problems , 2000 .

[26]  H. V. der Residual Replacement Strategies for Krylov Subspace Iterative Methods for the Convergence of True Residuals , 2000 .

[27]  Yves Robert,et al.  Parallel conjugate gradient-like algorithms for solving sparse nonsymmetric linear systems on a vector multiprocessor , 1989, Parallel Comput..

[28]  Henk A. van der Vorst,et al.  Parallel incomplete factorizations with pseudo-overlapped subdomains , 2001, Parallel Comput..

[29]  Takumi Washio,et al.  Parallel block preconditioning based on SSOR and MILU , 1994, Numer. Linear Algebra Appl..

[30]  I. Duff,et al.  The effect of ordering on preconditioned conjugate gradients , 1989 .

[31]  C.-C. Jay Kuo,et al.  Two-Color Fourier Analysis of Iterative Algorithms for Elliptic Problems with Red/Black Ordering , 1990, SIAM J. Sci. Comput..

[32]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[33]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .

[34]  T. Manteuffel,et al.  Necessary and Sufficient Conditions for the Existence of a Conjugate Gradient Method , 1984 .

[35]  Howard C. Elman,et al.  Relaxed and stabilized incomplete factorizations for non-self-adjoint linear systems , 1989 .

[36]  G. Meurant Computer Solution of Large Linear Systems , 1999 .

[37]  Gene H. Golub,et al.  Matrix computations , 1983 .

[38]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[39]  Michael Jung,et al.  Parallel Solvers for Nonlinear Elliptic Problems Based on Domain Decomposition Ideas , 1997, Parallel Comput..

[40]  Gérard Meurant Practical use of the conjugate gradient method on parallel supercomputers , 1989 .

[41]  H. V. D. Vorst,et al.  High Performance Preconditioning , 1989 .

[42]  Henk A. van der Vorst,et al.  A parallel linear system solver for circuit simulation problems , 2000, Numer. Linear Algebra Appl..

[43]  I. Gustafsson A class of first order factorization methods , 1978 .