Performance Optimizations and Bounds for Sparse Matrix Kernels

Building high-performance implementations of sparse matrix-vector multiply (SpM×V), an important and ubiquitous computational kernel, is fundamentally limited by a variety of factors: the increasing performance gap between processors and memory, the storage and instruction overhead of manipulating sparse data structures, and the irregular memory access due to sparse storage. Moreover, the complexity of modeling execution of modern microprocessors makes selecting the best data structure and SpM×V implementation for a given sparse matrix a difficult task. In this paper, we consider a range of performance bounds and models for SpM×V, both practical and hypothetical, which quantify these limits. The models vary both in cost and by what information is assumed— from the purely static to those that assume perfect knowledge of run-time information available via processor hardware counters. We evaluate these models and bounds on a variety of hardware platforms and matrices, and show that the task of selecting the best implementation and data structure for a particular matrix will require a combination of modeling and runtime searching. Furthermore, we use our performance bounds to show that our previously developed optimization technique, register blocking, which improves register-level reuse by exploiting naturally occurring dense subblocks, is unlikely to be improved upon significantly by additional low-level instruction scheduling efforts. Instead, we examine our recent efforts to overcome the fundamental limits through the use other kernels: multiplication with symmetric matrices, by multiple vectors, and higher-level kernels like multiplication of a vector by AA. ∗CS Division and Department of Mathematics