Representing linear algebra algorithms in code: the FLAME application program interfaces

In this article, we present a number of Application Program Interfaces (APIs) for coding linear algebra algorithms. On the surface, these APIs for the MATLAB M-script and C programming languages appear to be simple, almost trivial, extensions of those languages. Yet with them, the task of programming and maintaining families of algorithms for a broad spectrum of linear algebra operations is greatly simplified. In combination with our Formal Linear Algebra Methods Environment (FLAME) approach to deriving such families of algorithms, dozens of algorithms for a single linear algebra operation can be derived, verified to be correct, implemented, and tested, often in a matter of minutes per algorithm. Since the algorithms are expressed in code much like they are explained in a classroom setting, these APIs become not just a tool for implementing libraries, but also a valuable tool for teaching the algorithms that are incorporated in the libraries. In combination with an extension of the Parallel Linear Algebra Package (PLAPACK) API, the approach presents a migratory path from algorithm to MATLAB implementation to high-performance sequential implementation to parallel implementation. Finally, the APIs are being used to create a repository of algorithms and implementations for linear algebra operations, the FLAME Interface REpository (FIRE), which already features hundreds of algorithms for dozens of commonly encountered linear algebra operations.

[1]  John A. Gunnels,et al.  A systematic approach to the design and analysis of linear algebra algorithms , 2001 .

[2]  Robert A. van de Geijn,et al.  FLAME: Formal Linear Algebra Methods Environment , 2001, TOMS.

[3]  Brian W. Kernighan,et al.  The C Programming Language , 1978 .

[4]  Charles L. Lawson,et al.  Basic Linear Algebra Subprograms for Fortran Usage , 1979, TOMS.

[5]  Robert A. van de Geijn,et al.  Formal derivation of algorithms: The triangular sylvester equation , 2003, TOMS.

[6]  Robert A. van de Geijn,et al.  Formal Methods for High-Performance Linear Algebra Libraries , 2000, The Architecture of Scientific Software.

[7]  Robert A. van de Geijn,et al.  PLAPACK: high performance through high-level abstraction , 1998, Proceedings. 1998 International Conference on Parallel Processing (Cat. No.98EX205).

[8]  Robert A. van de Geijn,et al.  Developing Linear Algebra Algorithms: A Collection of Class Projects , 2001 .

[9]  Calvin Lin,et al.  Broadway: A Software Architecture for Scientific Computing , 2000, The Architecture of Scientific Software.

[10]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[11]  Calvin Lin,et al.  An annotation language for optimizing software libraries , 1999, DSL '99.

[12]  William Gropp,et al.  PETSc 2.0 users manual , 2000 .

[13]  Robert A. van de Geijn,et al.  A Parallel Eigensolver for Dense Symmetric Matrices Based on Multiple Relatively Robust Representations , 2005, SIAM J. Sci. Comput..

[14]  Jack J. Dongarra,et al.  Basic Linear Algebra Subprograms Technical (Blast) Forum Standard (1) , 2002, Int. J. High Perform. Comput. Appl..

[15]  Calvin Lin,et al.  Customizing Software Libraries for Performance Portability , 2001, PPSC.

[16]  Ed Anderson,et al.  LAPACK Users' Guide , 1995 .

[17]  Jack J. Dongarra,et al.  A set of level 3 basic linear algebra subprograms , 1990, TOMS.

[18]  G. Stewart.  Introduction to matrix computations , 1973 .

[19]  Nicholas J. Higham,et al.  Accuracy and stability of numerical algorithms, Second Edition , 2002 .

[20]  Robert A. van de Geijn,et al.  The science of deriving dense linear algebra algorithms , 2005, TOMS.

[21]  Calvin Lin,et al.  Optimizing the Use of High Performance Software Libraries , 2000, LCPC.

[22]  Robert A. van de Geijn,et al.  Using PLAPACK - parallel linear algebra package , 1997 .

[23]  Jack J. Dongarra,et al.  An extended set of FORTRAN basic linear algebra subprograms , 1988, TOMS.

[24]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[25]  Jack J. Dongarra,et al.  Solving linear systems on vector and shared memory computers , 1990 .

[26]  Robert A. van de Geijn,et al.  The Science of Programming High-Performance Linear Algebra Libraries , 2007 .

[27]  Jack Dongarra,et al.  ScaLAPACK: a scalable linear algebra library for distributed memory concurrent computers , 1992, [Proceedings 1992] The Fourth Symposium on the Frontiers of Massively Parallel Computation.

[28]  Robert A. van de Geijn,et al.  PLAPACK Parallel Linear Algebra Package Design Overview , 1997, International Conference on Software Composition.