A study of iterative methods for the solution of systems of linear equations on transputer networks

This thesis is concerned with the parallel, iterative solution of non-singular systems of linear equations, using networks of transputers. We provide an overview of a number of iterative methods designed to solve symmetric and nonsymmetric systems, with emphasis on variants of the Conjugate-Gradients method. These methods are usually used in conjunction with preconditioners, either to achieve convergence of the method for a given system or to accelerate the rate of convergence, or both. In this context we discuss the use of a highly parallel class of preconditioners, the polynomial preconditioners, in the solution of both symmetric and nonsymmetric systems. We show through the solution of a set of test problems that symmetric systems can be effectively solved using the Conjugate-Gradients method coupled with a polynomial preconditioner. Unfortunately for nonsymmetric systems the situation is rather obscure, none of the nonsymmetric iterative solvers studied give a consistently good performance on the set of test problems considered. However we give some experimental evidence that polynomial preconditioners may be used in the nonsymmetric case with some success, even if the conditions of convergence of such preconditioners are not satisfied. We describe the parallelization of some iterative methods along with polynomial preconditioners using rectangular meshes of transputers. Two distinct classes of algorithms are developed, one for generic (dense and sparse) systems and the other for those systems

[1]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[2]  Jiaoxun Kuang,et al.  A survey of AOR and TOR methods , 1988 .

[3]  Apostolos Hadjidimos,et al.  Accelerated overrelaxation method , 1978 .

[4]  C. Micchelli,et al.  Polynomial Preconditioners for Conjugate Gradient Calculations , 1983 .

[5]  J. Daniel.  The Conjugate Gradient Method for Linear and Nonlinear Operator Equations , 1967 .

[6]  Michael T. Heath,et al.  Parallel Algorithms for Matrix Computations , 1987 .

[7]  Courtenay T. Vaughan,et al.  Efficient Polynomial Preconditioning for the Conjugate Gradient Method , 1990, SIAM J. Sci. Comput..

[8]  T. Manteuffel,et al.  A taxonomy for conjugate gradient methods , 1990 .

[9]  Michael T. Heath,et al.  Modified cyclic algorithms for solving triangular systems on distributed-memory multiprocessors , 1988 .

[10]  H. V. D. Vorst,et al.  The rate of convergence of Conjugate Gradients , 1986 .

[11]  Inmos Limited,et al.  OCCAM 2 reference manual , 1988 .

[12]  Thomas F. Coleman,et al.  A New Method for Solving Triangular Systems on Distributed Memory Message-Passing Multiprocessors , 1989 .

[13]  G. Stewart.  The convergence of the method of conjugate gradients at isolated extreme points of the spectrum , 1975 .

[14]  Jack J. Dongarra,et al.  A set of level 3 basic linear algebra subprograms , 1990, TOMS.

[15]  Jagdish J. Modi,et al.  Parallel algorithms and matrix computation , 1988 .

[16]  Apostolos Hadjidimos,et al.  A survey of the iterative methods for the solution of linear systems by extrapolation, relaxation and other techniques , 1987 .

[17]  David J. Evans,et al.  Successive underrelaxation (SUR) and generalised conjugate gradient (GCG) methods for hyperbolic difference equations on a parallel computer , 1990, Parallel Comput..

[18]  Henk A. van der Vorst,et al.  A Vectorizable Variant of some ICCG Methods , 1982 .

[19]  I. Duff.  Sparse Matrices and Their Uses. , 1983 .

[20]  L. Adams.  m-Step Preconditioned Conjugate Gradient Methods , 1985 .

[21]  Y. Saad.  Krylov subspace methods for solving large unsymmetric linear systems , 1981 .

[22]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[23]  Jack J. Dongarra,et al.  An extended set of FORTRAN basic linear algebra subprograms , 1988, TOMS.

[24]  Y. Saad,et al.  Practical Use of Polynomial Preconditionings for the Conjugate Gradient Method , 1985 .

[25]  Jack J. Dongarra,et al.  Algorithm 656: an extended set of basic linear algebra subprograms: model implementation and test programs , 1988, TOMS.

[26]  Umberto Villano,et al.  Transputers and parallel architectures: message-passing distributed systems , 1991 .

[27]  David M. Young,et al.  On Richardson's Method for Solving Linear Systems with Positive Definite Matrices , 1953 .

[28]  S. Eisenstat,et al.  Variational Iterative Methods for Nonsymmetric Systems of Linear Equations , 1983 .

[29]  P. Sonneveld.  CGS, A Fast Lanczos-Type Solver for Nonsymmetric Linear systems , 1989 .

[30]  Thomas F. Coleman,et al.  A parallel triangular solver for distributed-memory multiprocessor , 1988 .

[31]  L. Richardson.  The Approximate Arithmetical Solution by Finite Differences of Physical Problems Involving Differential Equations, with an Application to the Stresses in a Masonry Dam , 1911 .

[32]  Gene H. Golub,et al.  Matrix computations , 1983 .

[33]  C. F. Schofield.  Optimising Fortran programs , 1989 .

[34]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .

[35]  Kang C. Jea,et al.  Generalized conjugate-gradient acceleration of nonsymmetrizable iterative methods , 1980 .

[36]  A. Greenbaum.  Comparison of splittings used with the conjugate gradient algorithm , 1979 .

[37]  Alan Edelman,et al.  The first annual large dense linear system survey , 1991, SGNM.

[38]  Paul E. Saylor,et al.  An adaptive algorithm for Richardson's method , 1990 .

[39]  C. A. R. Hoare,et al.  Communicating sequential processes , 1978, CACM.

[40]  Michael T. Heath,et al.  Parallel solution of triangular systems on distributed-memory multiprocessors , 1988 .

[41]  A. Jennings.  Influence of the Eigenvalue Spectrum on the Convergence Rate of the Conjugate Gradient Method , 1977 .

[42]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[43]  Tim Hopkins,et al.  The Parallel Solution of Partial Differential Equations on Transputer Networks , 1992 .

[44]  D. O’Leary.  Yet another polynomial preconditioner for the conjugate gradient algorithm , 1991 .

[45]  G. Smith,et al.  Numerical Solution of Partial Differential Equations: Finite Difference Methods , 1978 .

[46]  Apostolos Hadjidimos,et al.  On some extensions of the accelerated overrelaxation (AOR) theory , 1982 .

[47]  E. Di Zitti,et al.  High efficiency solution of triangular system equations on a 2-D array of transputers , 1989 .

[48]  Geoff Barrett,et al.  Formal Methods Applied to a Floating-Point Number System , 1989, IEEE Trans. Software Eng..

[49]  E. J. Craig.  The N‐Step Iteration Procedures , 1955 .

[50]  Tim Hopkins,et al.  The Parallel Solution of Triangular Systems of Linear Equations , 1991 .

[51]  Louis A. Hageman,et al.  Iterative Solution of Large Linear Systems. , 1971 .

[52]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .

[53]  Jack J. Dongarra,et al.  Algorithm 679: A set of level 3 basic linear algebra subprograms: model implementation and test programs , 1990, TOMS.

[54]  J. Pasciak,et al.  Computer solution of large sparse positive definite systems , 1982 .

[55]  V. B. Uvarov,et al.  Classical Orthogonal Polynomials of a Discrete Variable , 1991 .

[56]  Gene H. Golub,et al.  Some History of the Conjugate Gradient and Lanczos Algorithms: 1948-1976 , 1989, SIAM Rev..

[57]  R. Fletcher.  Conjugate gradient methods for indefinite systems , 1976 .

[58]  Rob H. Bisseling,et al.  Parallel Triangular System Solving on a Mesh Network of Transputers , 1991, SIAM J. Sci. Comput..

[59]  G. Golub,et al.  Iterative solution of linear systems , 1991, Acta Numerica.

[60]  Tim Hopkins,et al.  The Parallel Solution of Systems of Linear Equations using Iterative Methods on Transputer Networks , 1992 .

[61]  G. Temple,et al.  The General Theory of Relaxation Methods Applied to Linear Systems , 1939 .

[62]  J. Reid.  The Use of Conjugate Gradients for Systems of Linear Equations Possessing “Property A” , 1972 .

[63]  H. Elman.  Iterative methods for large, sparse, nonsymmetric systems of linear equations , 1982 .