Supercomputer Implementations of Preconditioned Krylov Subspace Methods

Preconditioned Krylov subspace methods are among the preferred iterative techniques for solving large sparse linear systems of equations. As computer architectures are evolving and problems are becoming more complex, iterative techniques are undergoing several mutations. In particular, there has been much recent work on new preconditioned that yield higher parallelism or on new implementations of the standard ones. In the past, a conservative and well understood approach has consisted of porting standard preconditioners to the new computers. However, in addition to their limited parallelism, such preconditioners have other drawbacks. The simple ILU(O) pre-conditioner may fail to converge for realistic problems arising from applications such as Computational Fluid Dynamics. The more robust analogues ILU(k) [30] or ILUT(k) [40] that allow more fill-in are not always a good alternative since they tend to be sequential in nature. In this paper we discuss these issues and give an overview of the standard approaches. Then we will propose a number of alternatives. It will be argued that some approaches based on multi-coloring can offer good compromises between generality and efficiency.

[1]  G. V. Paolini,et al.  Data structures to vectorize CG algorithms for general sparsity patterns , 1989 .

[2]  Niel K. Madsen,et al.  Matrix Multiplication by Diagonals on a Vector/Parallel Processor , 1976, Inf. Process. Lett..

[3]  Henk A. van der Vorst,et al.  Large tridiagonal and block tridiagonal linear systems on vector and parallel computers , 1987, Parallel Comput..

[4]  Thomas J. R. Hughes,et al.  LARGE-SCALE VECTORIZED IMPLICIT CALCULATIONS IN SOLID MECHANICS ON A CRAY X-MP/48 UTILIZING EBE PRECONDITIONED CONJUGATE GRADIENTS. , 1986 .

[5]  I. Duff,et al.  The effect of ordering on preconditioned conjugate gradients , 1989 .

[6]  J. Ortega Introduction to Parallel and Vector Solution of Linear Systems , 1988, Frontiers of Computer Science.

[7]  C. H. Wu A multicolour SOR method for the finite-element method , 1990 .

[8]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .

[9]  O. Axelsson A generalized conjugate gradient, least square method , 1987 .

[10]  I. Duff,et al.  Direct Methods for Sparse Matrices , 1987 .

[11]  H. J. Wirz Ill-posed design methods for the computation of inviscid flows , 1986 .

[12]  Kang C. Jea,et al.  Generalized conjugate-gradient acceleration of nonsymmetrizable iterative methods , 1980 .

[13]  Henk A. van der Vorst Analysis of a parallel solution method for tridiagonal linear systems , 1987, Parallel Comput..

[14]  J. Ortega,et al.  A multi-color SOR method for parallel computation , 1982, ICPP.

[15]  F. Shakib Finite element analysis of the compressible Euler and Navier-Stokes equations , 1989 .

[16]  R. Varga,et al.  Chebyshev semi-iterative methods, successive overrelaxation iterative methods, and second order Richardson iterative methods , 1961 .

[17]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[18]  V. Venkatakrishnan Preconditioned conjugate gradient methods for the compressible Navier-Stokes equations , 1990 .

[19]  G. Golub,et al.  ITERATIVE METHODS FOR CYCLICALLY REDUCED NON-SELF-ADJOINT LINEAR SYSTEMS , 1990 .

[20]  H. V. D. Vorst,et al.  High Performance Preconditioning , 1989 .

[21]  S. Eisenstat,et al.  An experimental study of methods for parallel preconditioned Krylov methods , 1989, C3P.

[22]  Thomas C. Oppe,et al.  NSPCG (Nonsymmetric Preconditioned Conjugate Gradient) user's guide: Version 1. 0: A package for solving large sparse linear systems by various iterative methods , 1988 .

[23]  Y. Saad,et al.  Conjugate gradient-like algorithms for solving nonsymmetric linear systems , 1985 .

[24]  Louis A. Hageman,et al.  Iterative Solution of Large Linear Systems. , 1971 .

[25]  Jack Dongarra,et al.  Proceedings of the Fourth SIAM Conference on Parallel Processing for Scientific Computing, Chicago, Illinois, USA, December 11-13, 1989 , 1990, PPSC.

[26]  Thomas C. Oppe,et al.  The performance of ITPACK on vector computers for solving large sparse linear systems arising in sample oil reseervoir simulation problems , 1987 .

[27]  P. E. Plassmann,et al.  Parallel iterative solution of sparse linear systems using orderings from graph coloring heuristics , 1990 .

[28]  L. Adams Iterative algorithms for large sparse linear systems on parallel computers , 1983 .

[29]  H. F. Jordan,et al.  Is SOR Color-Blind? , 1986 .

[30]  P. K. W. Vinsome,et al.  Orthomin, an Iterative Method for Solving Sparse Sets of Simultaneous Linear Equations , 1976 .

[31]  Joel H. Saltz,et al.  Automated problem scheduling and reduction of synchronization delay effects , 1987 .

[32]  Y. Saad Least squares polynomials in the complex plane and their use for solving nonsymmetric linear systems , 1987 .

[33]  Henk A. van der Vorst,et al.  The performance of FORTRAN implementations for preconditioned conjugate gradients on vector computers , 1986, Parallel Comput..

[34]  Youcef Saad,et al.  Parallel Implementations of Preconditioned Conjugate Gradient Methods. , 1985 .

[35]  Omar Wing,et al.  A Computation Model of Parallel Solution of Linear Equations , 1980, IEEE Transactions on Computers.

[36]  Yousef Saad,et al.  A Flexible Inner-Outer Preconditioned GMRES Algorithm , 1993, SIAM J. Sci. Comput..

[37]  Roland W. Freund,et al.  An Implementation of the Look-Ahead Lanczos Algorithm for Non-Hermitian Matrices , 1993, SIAM J. Sci. Comput..

[38]  Rami G. Melhem,et al.  Parallel solution of linear systems with striped sparse matrices , 1988, Parallel Comput..

[39]  R. Grimes,et al.  On vectorizing incomplete factorization and SSOR preconditioners , 1988 .

[40]  T. Manteuffel,et al.  Adaptive polynomial preconditioning for hermitian indefinite linear systems , 1989 .

[41]  E. L. Poole,et al.  Multicolor ICCG methods for vector computers , 1987 .

[42]  Youcef Saad,et al.  A Basic Tool Kit for Sparse Matrix Computations , 1990 .

[43]  T. Manteuffel Adaptive procedure for estimating parameters for the nonsymmetric Tchebychev iteration , 1978 .

[44]  H. Elman Iterative methods for large, sparse, nonsymmetric systems of linear equations , 1982 .

[45]  M. Heroux,et al.  A parallel preconditioned conjugate gradient package for solving sparse linear systems on a Cray Y-MP , 1991 .

[46]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[47]  T. Manteuffel The Tchebychev iteration for nonsymmetric linear systems , 1977 .