A Modern Framework for Portable High-Performance Numerical Linear Algebra

In this chapter, we present a generic programming methodology for expressing data structures and algorithms for numerical linear algebra. We also present a high-performance implementation of this approach, the Matrix Template Library (MTL). As with the Standard Template Library, our approach is five-fold, consisting of generic functions, containers, iterators, adapters, and function objects, all developed specifically for high-performance numerical linear algebra. Within our framework, we provide generic functions corresponding to the mathematical operations that define linear algebra. Similarly, the containers, adapters, and iterators are used to represent and to manipulate concrete linear algebra objects such as matrices and vectors. To many scientific computing users, however, the advantages of an elegant programming interface are secondary to issues of performance. There are two aspects to how we achieved high performance in the MTL. The first is the use of static polymorphism (template functions) with modern optimizing compilers, enabling extreme flexibility with no loss in performance. The second is the application of abstraction to the optimization process itself. The Basic Linear Algebra Instruction Set (BLAIS) is presented as an abstract interface to several important performance optimizations. Our experimental results show that MTL with the BLAIS achieves performance that is as good as, or better than, vendor-tuned libraries, even though MTL and the BLAIS are written completely in C++. We therefore conclude that the use of abstraction is not a barrier to performance, contrary to conventional wisdom, and that certain abstractions can in fact facilitate optimization. In addition, MTL requires orders of magnitude fewer lines of code for its implementation, with the concomitant savings in development and maintenance effort.