ScaLAPACK: A Portable Linear Algebra Library for Distributed Memory Computers - Design Issues and Performance

This paper outlines the content and performance of ScaLAPACK, a collection of mathematical software for linear algebra computations on distributed memory computers. The importance of developing standards for computational and message passing interfaces is discussed. We present the different components and building blocks of ScaLAPACK, and indicate the difficulties inherent in producing correct codes for networks of heterogeneous processors. Finally, this paper briefly describes future directions for the ScaLAPACK library and concludes by suggesting alternative approaches to mathematical libraries, explaining how ScaLAPACK could be integrated into efficient and user-friendly distributed systems.

[1]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[2]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[3]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[4]  Charles L. Lawson,et al.  A proposal for standard linear algebra subprograms , 1973 .

[5]  C. Lawson,et al.  Improving the efficiency of portable software for linear algebra , 1973, SGNM.

[6]  C. Paige.  Fast Numerically Stable Computations for Generalized Linear Least Squares Problems , 1979 .

[7]  Charles L. Lawson,et al.  Basic Linear Algebra Subprograms for Fortran Usage , 1979, TOMS.

[8]  Ed Anderson,et al.  LAPACK Users' Guide , 1995 .

[9]  Jack J. Dongarra,et al.  Solving banded systems on a parallel processor , 1987, Parallel Comput..

[10]  Jack Dongarra,et al.  ScaLAPACK Users' Guide , 1987 .

[11]  Jack J. Dongarra,et al.  Algorithm 656: an extended set of basic linear algebra subprograms: model implementation and test programs , 1988, TOMS.

[12]  G. C. Fox,et al.  Solving Problems on Concurrent Processors , 1988 .

[13]  Jack J. Dongarra,et al.  An extended set of FORTRAN basic linear algebra subprograms , 1988, TOMS.

[14]  Jack J. Dongarra,et al.  A set of level 3 basic linear algebra subprograms , 1990, TOMS.

[15]  Geoffrey C. Fox,et al.  Solving problems on concurrent processors: vol. 2 , 1990 .

[16]  Jack Dongarra,et al.  LAPACK Working Note 37: Two Dimensional Basic Linear Algebra Communication Subprograms , 1991 .

[17]  Bart De Moor,et al.  Generalizations of the singular value and QR decompositions , 1991, Signal Process..

[18]  J. Dongarra,et al.  Generalized QR factorization and its applications , 1992 .

[19]  R. van de Geijn,et al.  A look at scalable dense linear algebra libraries , 1992, Proceedings Scalable High Performance Computing Conference SHPCC-92..

[20]  Jack Dongarra,et al.  ScaLAPACK: a scalable linear algebra library for distributed memory concurrent computers , 1992, [Proceedings 1992] The Fourth Symposium on the Frontiers of Massively Parallel Computation.

[21]  Rice UniversityCORPORATE,et al.  High performance Fortran language specification , 1993 .

[22]  Corporate Rice University,et al.  High performance Fortran language specification , 1993, FORF.

[23]  Jack Dongarra,et al.  Parallel matrix transpose algorithms on distributed memory concurrent computers , 1993, Proceedings of Scalable Parallel Libraries Conference.

[24]  Robert A. van de Geijn,et al.  Two Dimensional Basic Linear Algebra Communication Subprograms , 1993, PPSC.

[25]  Jack Dongarra,et al.  The design of linear algebra libraries for high performance computers , 1993 .

[26]  Jack Dongarra,et al.  LAPACK Working Note 58: ``The Design of Linear Algebra Libraries for High Performance Computers , 1993 .

[27]  James Demmel,et al.  Design of a Parallel Nonsymmetric Eigenroutine Toolbox, Part I , 1993, PPSC.

[28]  Guy L. Steele,et al.  The High Performance Fortran Handbook , 1993 .

[29]  J. Demmel,et al.  LAPACK Working Note 86: The Performance of Finding Eigenvalues and Eigenvectors of Dense Symmetric Matrices on Distributed Memory Computers , 1994 .

[30]  Message P Forum,et al.  MPI: A Message-Passing Interface Standard , 1994 .

[31]  James Demmel,et al.  LAPACK Working Note 70: On the Correctness of Parallel Bisection in Floating Point , 1994 .

[32]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[33]  Jack J. Dongarra,et al.  Software Libraries for Linear Algebra Computations on High Performance Computers , 1995, SIAM Rev..

[34]  Jaeyoung Choi,et al.  Parallel Matrix Transpose Algorithms on Distributed Memory Concurrent Computers , 1995, Parallel Comput..

[35]  R. C. Whaley,et al.  LAPACK Working Note 94: A User''s Guide to the BLACS v1.0 , 1995 .

[36]  Jack Dongarra,et al.  PVM: Parallel virtual machine: a users' guide and tutorial for networked parallel computing , 1995 .

[37]  James Demmel,et al.  The Performance of Finding Eigenvalues and Eigenvaectors of Dense Symmetric Matrices on Distributed Memory Computers , 1995, PPSC.

[38]  James Demmel,et al.  Modeling the benefits of mixed data and task parallelism , 1995, SPAA '95.

[39]  Jack Dongarra,et al.  LAPACK Working Note 91: The Spectral Decomposition of Nonsymmetric Matrices on Distributed Memory Parallel Computers , 1995 .

[40]  Jaeyoung Choi,et al.  A Proposal for a Set of Parallel Basic Linear Algebra Subprograms , 1995, PARA.

[41]  James Demmel,et al.  ScaLAPACK: A Portable Linear Algebra Library for Distributed Memory Computers - Design Issues and Performance , 1995, PARA.

[42]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[43]  James Demmel,et al.  Practical Experience in the Dangers of Heterogeneous Computing , 1996, PARA.

[44]  Jack Dongarra,et al.  The dangers of heterogeneous network computing: heterogeneous networks considered harmful , 1996 .

[45]  Jaeyoung Choi,et al.  Design and Implementation of the ScaLAPACK LU, QR, and Cholesky Factorization Routines , 1994, Sci. Program..

[46]  J. Demmel,et al.  An inverse free parallel spectral divide and conquer algorithm for nonsymmetric eigenproblems , 1997 .