ScaLAPACK: A Portable Linear Algebra Library for Distributed Memory Computers - Design Issues and Performance (Technical Paper)

This paper outlines the content and performance of ScaLAPACK, a collection of mathematical software for linear algebra computations on distributed memory computers. The importance of developing standards for computational and message passing interfaces is discussed. We present the different components and building blocks of ScaLAPACK, and indicate the difficulties inherent in producing correct codes for networks of heterogeneous processors. Finally, this paper briefly describes future directions for the ScaLAPACK library and concludes by suggesting alternative approaches to mathematical libraries, explaining how ScaLAPACK could be integrated into efficient and user-friendly distributed systems.

[1]  Message P Forum,et al.  MPI: A Message-Passing Interface Standard , 1994 .

[2]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[3]  Petter E. Bjørstad,et al.  Proceedings of the Seventh SIAM Conference on Parallel Processing for Scientific Computing, PPSC 1995, San Francisco, California, USA, February 15-17, 1995 , 1995, PPSC.

[4]  J. Demmel,et al.  An inverse free parallel spectral divide and conquer algorithm for nonsymmetric eigenproblems , 1997 .

[5]  Jack Dongarra,et al.  ScaLAPACK: a scalable linear algebra library for distributed memory concurrent computers , 1992, [Proceedings 1992] The Fourth Symposium on the Frontiers of Massively Parallel Computation.

[6]  C. Paige.  Fast Numerically Stable Computations for Generalized Linear Least Squares Problems , 1979 .

[7]  Jack J. Dongarra,et al.  Solving banded systems on a parallel processor , 1987, Parallel Comput..

[8]  Charles L. Lawson,et al.  A proposal for standard linear algebra subprograms , 1973 .

[9]  Rice UniversityCORPORATE,et al.  High performance Fortran language specification , 1993 .

[10]  Jack Dongarra,et al.  The design of linear algebra libraries for high performance computers , 1993 .

[11]  James Demmel,et al.  Design of a Parallel Nonsymmetric Eigenroutine Toolbox, Part I , 1993, PPSC.

[12]  G. C. Fox,et al.  Solving Problems on Concurrent Processors , 1988 .

[13]  Jack Dongarra,et al.  LAPACK Working Note 91: The Spectral Decomposition of Nonsymmetric Matrices on Distributed Memory Parallel Computers , 1995 .

[14]  Bart De Moor,et al.  Generalizations of the Singular Value and QR-Decompositions , 1992, SIAM J. Matrix Anal. Appl..

[15]  Jack J. Dongarra,et al.  An extended set of FORTRAN basic linear algebra subprograms , 1988, TOMS.

[16]  James Demmel,et al.  The Performance of Finding Eigenvalues and Eigenvaectors of Dense Symmetric Matrices on Distributed Memory Computers , 1995, PPSC.

[17]  Jaeyoung Choi,et al.  A Proposal for a Set of Parallel Basic Linear Algebra Subprograms , 1995, PARA.

[18]  James Demmel,et al.  ScaLAPACK: A Portable Linear Algebra Library for Distributed Memory Computers - Design Issues and Performance , 1995, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[19]  J. Dongarra,et al.  Generalized QR factorization and its applications , 1992 .

[20]  Jack J. Dongarra,et al.  A set of level 3 basic linear algebra subprograms , 1990, TOMS.

[21]  James Demmel,et al.  Practical Experience in the Dangers of Heterogeneous Computing , 1996, PARA.

[22]  Guy L. Steele,et al.  The High Performance Fortran Handbook , 1993 .

[23]  Jack Dongarra,et al.  The dangers of heterogeneous network computing: heterogeneous networks considered harmful , 1996 .

[24]  Charles L. Lawson,et al.  Basic Linear Algebra Subprograms for Fortran Usage , 1979, TOMS.

[25]  James Demmel,et al.  Modeling the benefits of mixed data and task parallelism , 1995, SPAA '95.

[26]  Jack Dongarra,et al.  PVM: Parallel virtual machine: a users' guide and tutorial for networked parallel computing , 1995 .

[27]  Jack J. Dongarra,et al.  Algorithm 656: an extended set of basic linear algebra subprograms: model implementation and test programs , 1988, TOMS.