Solution of Partial Differential Equations on Vector and Parallel Computers

In this work we review the present status of numerical methods for partial differential equations on vector and parallel computers. A discussion of the relevant aspects of these computers and a brief review of their development is included, with particular attention paid to those characteristics that influence algorithm selection. Both direct and iterative methods are given for elliptic equations as well as explicit and implicit methods for initial-boundary value problems. The intent is to point out attractive methods as well as areas where this class of computer architecture cannot be fully utilized because of either hardware restrictions or the lack of adequate algorithms. A brief discussion of application areas utilizing these computers is included.

[1]  J. H. Wilkinson The calculation of the latent roots and vectors of matrices on the pilot model of the A.C.E. , 1954, Mathematical Proceedings of the Cambridge Philosophical Society.

[2]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[3]  Jürg Nievergelt,et al.  Parallel methods for integrating ordinary differential equations , 1964, CACM.

[4]  Marshall C. Pease Matrix Inversion Using Parallel Processing , 1967, JACM.

[5]  W. Miranker,et al.  Parallel methods for the numerical integration of ordinary differential equations , 1967 .

[6]  Marshall C. Pease,et al.  An Adaptation of the Fast Fourier Transform for Parallel Processing , 1968, JACM.

[7]  Jack L. Rosenfeld,et al.  A case study in programming for parallel-processors , 1969, CACM.

[8]  W. Miranker A Survey of Parallelism in Numerical Analysis , 1971 .

[9]  A. Sameh On Jacobi and Jacobi-like algorithms for a parallel computer , 1971 .

[10]  Harold S. Stone,et al.  Parallel Processing with the Perfect Shuffle , 1971, IEEE Transactions on Computers.

[11]  Louis A. Hageman,et al.  Iterative Solution of Large Linear Systems. , 1971 .

[12]  Willis H. Ware,et al.  The ultimate computer , 1972, IEEE Spectrum.

[13]  W. J. Watson The TI ASC: a highly modular and flexible super computer architecture , 1972, AFIPS '72 (Fall, part I).

[14]  Jack A. Rudolph A production implementation of an associative array processor: STARAN , 1972, AFIPS '72 (Fall, part I).

[15]  J. Reid The Use of Conjugate Gradients for Systems of Linear Equations Possessing “Property A” , 1972 .

[16]  Peter M. Kogge Maximal rate pipelined solutions to recurrence problems , 1973, ISCA 1973.

[17]  Harold S. Stone,et al.  A Parallel Algorithm for the Efficient Solution of a General Class of Recurrence Equations , 1973, IEEE Transactions on Computers.

[18]  Raymond E. Miller,et al.  A Comparison of Some Theoretical Models of Parallel Computation , 1973, IEEE Transactions on Computers.

[19]  A parallel algorithm for high subsonic compressible flow over a circular cylinder , 1973 .

[20]  Harold S. Stone,et al.  An Efficient Parallel Algorithm for the Solution of a Tridiagonal Linear System of Equations , 1973, JACM.

[21]  John Palmer,et al.  A fast method for solving a class of tridiagonal linear systems , 1974, Commun. ACM.

[22]  Peter M. Kogge,et al.  Parallel Solution of Recurrence Problems , 1974, IBM J. Res. Dev..

[23]  Ii John Franklin Palmer Conjugate-direction methods and parallel-computing. , 1974 .

[24]  Robert B. Wilhelmson Solving partial differential equations using ILLIAC IV , 1974 .

[25]  K. H. Coats,et al.  Direct methods in reservoir simulation , 1974 .

[26]  Harold S. Stone,et al.  Parallel Tridiagonal Equation Solvers , 1975, TOMS.

[27]  William G. Poole,et al.  System balance analysis for vector computers , 1975, ACM '75.

[28]  Jr. Jules Joseph Lambiotte The solution of linear systems of equations on a vector computer. , 1975 .

[29]  David J. Kuck,et al.  Parallel Processing of Ordinary Programs , 1976, Adv. Comput..

[30]  Niel K. Madsen,et al.  Matrix Multiplication by Diagonals on a Vector/Parallel Processor , 1976, Inf. Process. Lett..

[31]  Kenneth J. Thurber Large scale computer architecture: Parallel and associative processors , 1976 .

[32]  K. Fong,et al.  Some linear algebraic algorithms and their performance on CRAY-1 , 1977 .

[33]  A large scale, homogeneous, fully distributed parallel machine, I , 1977, ISCA '77.

[34]  P. Swarztrauber THE METHODS OF CYCLIC REDUCTION, FOURIER ANALYSIS AND THE FACR ALGORITHM FOR THE DISCRETE SOLUTION OF POISSON'S EQUATION ON A RECTANGLE* , 1977 .

[35]  J. Meijerink,et al.  An iterative solution method for linear systems of which the coefficient matrix is a symmetric -matrix , 1977 .

[36]  R. Brent,et al.  Solving Triangular Systems on a Parallel Computer , 1977 .

[37]  David J. Kuck,et al.  A Survey of Parallel Machine Organization and Programming , 1977, CSUR.

[38]  David J. Kuck,et al.  A Parallel QR Algorithm for Symmetric Tridiagonal Matrices , 1977, IEEE Transactions on Computers.

[39]  Yehuda Wallach,et al.  Iterative Solution of Linear Equations on a Parallel Processor System , 1977, IEEE Transactions on Computers.

[40]  Duncan H. Lawrie,et al.  High Speed Computer and Algorithm Organization , 1977 .

[41]  Ahmed Sameh,et al.  Parallel direct linear system solvers -- a survey☆ , 1977 .

[42]  Franco P. Preparata,et al.  An Improved Parallel Processor Bound in Fast Matrix Inversion , 1978, Inf. Process. Lett..

[43]  W. L. Miranker,et al.  Parallel methods for solving equations , 1978 .

[44]  Takayuki Kimura,et al.  Decentralized parallel algorithms for matrix computation , 1978, ISCA '78.

[45]  Robert E. Fulton,et al.  Substructuring techniques—status and projections , 1978 .

[46]  Levy Raskin Performance evaluation of multiple processor systems. , 1978 .

[47]  David L. Kuck,et al.  The Structure of Computers and Computations , 1978 .

[48]  H. T. Kung Let's Design Algorithms for VLSI Systems , 1979 .

[49]  J. T. Robinson,et al.  Some Analysis Techniques for Asynchronous Multiprocessor Algorithms , 1979, IEEE Transactions on Software Engineering.

[50]  George W. Platzman,et al.  The ENIAC Computations of 1950—Gateway to Numerical Weather Prediction , 1979 .

[51]  J. J. Lambiotte,et al.  Computing the Fast Fourier Transform on a vector computer , 1979 .

[52]  Philip C. Treleaven,et al.  Exploitng Program Concurrency in Computing Systems , 1979, Computer.

[53]  J. I. Pitts,et al.  The solution of the three-dimensional viscous-compressible Navier-Stokes equations on a vector computer , 1979 .

[54]  Clive Temperton,et al.  Direct methods for the solution of the discrete Poisson equation: Some comparisons , 1979 .

[55]  Pamela Zave,et al.  Design of an Adaptive, Parallel Finite-Element System , 1979, TOMS.

[56]  Michael J. Kascic Vector Processing on the Cyber 200 and Vector Numerical Linear Algebra , 1980 .

[57]  Douglas Stott Parker,et al.  Notes on Shuffle/Exchange-Type Switching Networks , 1980, IEEE Transactions on Computers.

[58]  Yehuda Wallach,et al.  On Block-Parallel Methods for Solving Linear Equations , 1980, IEEE Transactions on Computers.

[59]  H. T. Kung The Structure of Parallel Algorithms , 1980, Adv. Comput..

[60]  G. Jack Lipovski,et al.  An overview of the Texas reconfigurable array computer , 1899, AFIPS '80.

[61]  Larry D. Wittie,et al.  MICROS, A Distributed Operating System for MICRONET, A Reconfigurable Network Computer , 1980, IEEE Trans. Computers.

[62]  Jerry C. South,et al.  Vector processor algorithms for transonic flow calculations , 1980 .

[63]  M. J. Kascic,et al.  Vector Processing on the CYBER 200 , 1980, Angew. Inform..

[64]  James D. Lee Three dimensional finite element analysis of layered fiber-reinforced composite materials , 1980 .

[65]  Y. Saad,et al.  A PARALLEL BLOCK STIEFEL METHOD FOR SOLVING POSITIVE DEFINITE SYSTEMS , 1981 .

[66]  Donald MacDavid Tolle,et al.  On the Complexity of Vector Computations in Binary Tree Machines , 1981, Inf. Process. Lett..

[67]  G. Rodrigue,et al.  Incomplete block cyclic reduction , 1981 .

[68]  H. H. Wang,et al.  A Parallel Method for Tridiagonal Equations , 1981, TOMS.

[69]  Ahmed H. Sameh,et al.  Iterative methods for the solution of elliptic difference equations on multiprocessors , 1981, CONPAR.

[70]  Martin H. Schultz,et al.  Elliptic problem solvers , 1981 .

[71]  Tilak Agerwala,et al.  Communication Issues in the Design and Analysis of Parallel Algorithms , 1981, IEEE Transactions on Software Engineering.

[72]  Peter M. Kogge,et al.  The Architecture of Pipelined Computers , 1981 .

[73]  Franco P. Preparata,et al.  The cube-connected-cycles: A versatile network for parallel computation , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[74]  W. James,et al.  A Conjugate Gradient-Truncated Direct Method for the Iterative Solution of the Reservoir Simulation Pressure Equation , 1981 .

[75]  J. Meijerink,et al.  Guidelines for the usage of incomplete decompositions in solving sets of linear equations as they occur in practical problems , 1981 .

[76]  Yukio Kaneda,et al.  Highly Parallel Computing of Linear Equations on the Matrix-Broadcast-Memory Connected Array Processor System , 1982, IMACS World Congress.

[77]  Swarn P. Kumar,et al.  Parallel algorithms for solving linear equations on mimd computers , 1982 .

[78]  Willi Schönauer,et al.  A Polyalgorithm with Diagonal Storing for the Solution of Very Large Indefinite Linear Banded Systems on a Vector Computer , 1982, IMACS World Congress.

[79]  H. T. Kung Why systolic architectures? , 1982, Computer.

[80]  T. Moto-Oka,et al.  Fifth Generation Computer Systems , 1982 .

[81]  G. Rodrigue Parallel Computations , 1982 .

[82]  Ian Watson,et al.  A Practical Data Flow Computer , 1982, Computer.

[83]  Paul N. Swarztrauber,et al.  Vectorizing the FFTs , 1982 .

[84]  Neil R. Lincoln Technology and Design Tradeoffs in the Creation of a Modern Supercomputer , 1982, IEEE Transactions on Computers.

[85]  G. Rodrigue,et al.  An Implicit Numerical Solution of the Two-Dimensional Diffusion Equation and Vectorization Experiments , 1982 .

[86]  Rami Melhem Formal Verification of a Systolic System for Finite Element Stiffness Matrices. , 1982 .

[87]  J. Philip Drummond,et al.  Numerical Study of Staged Fuel Injection for Supersonic Combustion , 1982 .

[88]  Seymour V. Parter,et al.  Block Iterative Methods for Elliptic and Parabolic Difference Equations. , 1982 .

[89]  Duncan H. Lawrie,et al.  A Practical Algorithm for the Solution of Triangular Systems on a Parallel Processing System , 1982, IEEE Transactions on Computers.

[90]  G. S. Pawley,et al.  The implementation of lattice calculations on the DAP , 1982 .

[91]  P. L. Stanat,et al.  Performance Comparisons for Reservoir Simulation Problems on Three Supercomputers , 1982 .

[92]  Gyula A. Magó,et al.  Solving Partial Differential Equations on a Cellular Tree Machine , 1982, IMACS World Congress.

[93]  Olaf O. Storaasli,et al.  the finite element machine: An experiment in parallel processing , 1982 .

[94]  H. T. Kung,et al.  Integrating High-Performance Special Purpose Devices Into A System , 1982, Other Conferences.

[95]  James G. Welsh Geophysical Fluid Simulation on a Parallel Computer , 1982 .

[96]  The solution of linear equations on a SIMD computer using a parallel iterative algorithm , 1982 .

[97]  David J. Kuck,et al.  The Burroughs Scientific Processor (BSP) , 1982, IEEE Transactions on Computers.

[98]  David S. Kershaw,et al.  Solution of Single Tridiagonal Linear Systems and Vectorization of the ICCG Algorithm on the Cray-1 , 1982 .

[99]  Lawrence Snyder,et al.  Introduction to the configurable, highly parallel computer , 1982, Computer.

[100]  Dennis Parkinson,et al.  The Measurement of Performance on a Highly Parallel System , 1983, IEEE Transactions on Computers.

[101]  Daniel Weaver,et al.  Performance of a Modular Interactive Data Analysis System (MIDAS) , 1983, ICPP.

[102]  M. Salama,et al.  Finite element computation with parallel VLSI , 1983 .

[103]  Nadia Y. Yousif Parallel algorithms for asynchronous multiprocessors , 1983 .

[104]  Yakup Paker Multi-Microprocessor Systems , 1983 .

[105]  W. P. Petersen,et al.  Vector Fortran for numerical problems on CRAY-1 , 1983, CACM.

[106]  John C. Knight,et al.  On the design of a special‐purpose scientific programming language , 1983, Softw. Pract. Exp..

[107]  A fully vectorized numerical solution of the incompressible Navier-Stokes equations. Ph.D. Thesis , 1983 .

[108]  N. D. Melson,et al.  EXPERIENCES IN USING THE CYBER 203 AND CYBER 205 FOR THREE-DIMENSIONAL TRANSONIC FLOW CALCULATIONS , 1983 .

[109]  R G Melhem An Abstract Systolic Model and Its Application to the Design of Finite Element Systems. , 1983 .

[110]  Neil R. Lincoln Supercomputers = Colossal Computations + Enormous Expectations + Renowned Risk , 1983, Computer.

[111]  R. E. Fulton,et al.  Impact of new computing systems on finite element computations , 1983 .

[112]  Doyle Knight,et al.  A hybrid explicit-implicit numerical algorithm for the three-dimensional compressible Navier-Stokes equations , 1983 .

[113]  Kai Hwang,et al.  Pipelined Evaluation of First-Order Recurrence Systems , 1983, ICPP.

[114]  Anthony T. Patera,et al.  Secondary instability of wall-bounded shear flows , 1983, Journal of Fluid Mechanics.

[115]  Thomas C. Oppe,et al.  Itpack on supercomputers , 1983 .

[116]  Dennis Parkinson The Distributed Array Processor (DAP) , 1983 .

[117]  Robert A. Wagner,et al.  The Boolean Vector Machine [BVM] , 1983, ISCA '83.

[118]  Senol Utku,et al.  Parallel solution of finite element equations , 1983 .

[119]  Duncan H. Lawrie,et al.  The computation and communication complexity of a parallel banded system solver , 1984, TOMS.

[120]  Jaques Tiberghien New Computer Architectures , 1984 .

[121]  Vasilii Zakharov Parallelism and Array Processing , 1984, IEEE Transactions on Computers.

[122]  D. A. Reed,et al.  A model of asynchronous iterative algorithms for solving large, sparse, linear systems , 1984 .

[123]  D. O’Leary Ordering Schemes for Parallel Processing of Certain Mesh Problems , 1984 .

[124]  Robert Schreiber SYSTOLIC ARRAYS: HIGH PERFORMANCE PARALLEL MACHINES FOR MATRIX COMPUTATION , 1984 .

[125]  Rami G. Melhem,et al.  A Mathematical Model for the Verification of Systolic Networks , 1984, SIAM J. Comput..

[126]  T. A. Zang,et al.  Spectral Methods for Partial Differential Equations , 1984 .

[127]  Udo Schendel,et al.  Introduction to Numerical Methods for Parallel Computers , 1984 .

[128]  G. Rodrigue,et al.  Preconditioning by incomplete block cyclic reduction , 1984 .

[129]  J. J. Modi,et al.  An alternative givens ordering , 1984 .

[130]  A. H. Sameh A FAST POISSON SOLVER FOR MULTIPROCESSORS , 1984 .

[131]  J. C. Browne,et al.  Techniques for Solving Block Tridiagonal Systems on Reconfigurable Array Computers , 1984 .

[132]  A. Kasahara RECENT MATHEMATICAL AND COMPUTATIONAL DEVELOPMENTS IN NUMERICAL WEATHER PREDICTION , 1984 .

[133]  C. T. Wu,et al.  Navier-Stokes Simulation of Homogeneous Turbulence on the CYBER 205 , 1984 .

[134]  Solving elliptic boundary-value problems on parallel processors by approximate inverse matrix semi-direct methods based on the multiple explicit Jacobi iteration , 1984 .

[135]  T. Jordan CONJUGATE GRADIENT PRECONDITIONERS FOR VECTOR AND PARALLEL PROCESSORS , 1984 .

[136]  Charles L. Seitz,et al.  Engineering limits on computer performance , 1984 .

[137]  John L. Larson Multitasking on the Cray X-MP-2 Multiprocessor , 1984, Computer.

[138]  Olof Widlund Iterative methods for elliptic problems on regions partitioned into substructures and the biharmonic Dirichlet problem , 1984 .

[139]  Martin H. Schultz SOLVING ELLIPTIC PROBLEMS ON AN ARRAY PROCESSOR SYSTEM , 1984 .

[140]  Daniel A. Reed,et al.  Parallel, iterative solution of sparse linear systems: Models and architectures , 1985, Parallel Comput..

[141]  Hiroshi Kashiwagi The Japanese super-speed computer project , 1985, Future Gener. Comput. Syst..

[142]  Efficiency of D4 Gaussian Elimination on a vector computer , 1985 .

[143]  Y. Saad,et al.  Practical Use of Polynomial Preconditionings for the Conjugate Gradient Method , 1985 .

[144]  G. Golub,et al.  Block Preconditioning for the Conjugate Gradient Method , 1985 .

[145]  Michael R. Leuze,et al.  Parallel triangularization of substructured finite element problems , 1986 .

[146]  D J Evans,et al.  Parallel processing , 1986 .

[147]  Duncan A. Buell Supercomputers and parallel computation , 1986 .

[148]  Seymour V. Parter Large Scale Scientific Computation. , 1986 .

[149]  Charles L. Seitz Experiments with VLSI ensemble machines , 1986 .

[150]  Marvin Minsky,et al.  Form and Content in Computer Science , 1987 .

[151]  Nicholas M. Baran Finite Element Analysis on Microcomputers , 1988 .