A GEMM interface and implementation on NVIDIA GPUs for multiple small matrices

We present an interface and an implementation of the General Matrix Multiply (GEMM) routine for multiple small matrices processed simultaneously on NVIDIA graphics processing units (GPUs). We focus on matrix sizes under 16. The implementation can be easily extended to larger sizes. For single precision matrices, our implementation is 30% to 600% faster than the batched cuBLAS implementation distributed in the CUDA Toolkit 5.0 on NVIDIA Tesla K20c. For example, we obtain 104 GFlop/s and 216 GFlop/s when multiplying 100,000 independent matrix pairs of size 10 and 16, respectively. Similar improvement in performance is obtained for other sizes, in single and double precisions for real and complex types, and when the number of matrices is smaller. Apart from our implementation, our different function interface also plays an important role in the improved performance. Applications of this software include finite element computation on GPUs. A second leading dimension-based batched GEMM interface for CUDA.Implementation of GEMM routine for multiple small matrices.30% to 600% faster than the batched cuBLAS in CUDA Toolkit 5.0.Specialized for matrix sizes under 16 on NVIDIA Tesla K20c.

[1]  Robert A. van de Geijn,et al.  High-performance implementation of the level-3 BLAS , 2008, TOMS.

[2]  Bjarne Stroustrup,et al.  The C++ Programming Language: Special Edition , 2000 .

[3]  Bo Kågström,et al.  GEMM-based level 3 BLAS: high-performance model implementations and performance evaluation benchmark , 1998, TOMS.

[4]  Bjarne Stroustrup,et al.  The C++ Programming Language, 4th Edition , 2013 .

[5]  Jack J. Dongarra,et al.  An Improved Magma Gemm For Fermi Graphics Processing Units , 2010, Int. J. High Perform. Comput. Appl..

[6]  Maciej Paszyński,et al.  Computing with hp-ADAPTIVE FINITE ELEMENTS: Volume II Frontiers: Three Dimensional Elliptic and Maxwell Problems with Applications , 2007 .

[7]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[8]  I. Doležel,et al.  Higher-Order Finite Element Methods , 2003 .

[9]  Victor Eijkhout,et al.  Sparse direct factorizations through unassembled hyper-matrices , 2010 .

[10]  Mads Torgersen,et al.  The c# programming language, third edition , 2008 .

[11]  Jack Dongarra,et al.  LAPACK Users' Guide, 3rd ed. , 1999 .

[12]  Joel Ferziger,et al.  Higher Order Methods for Incompressible Fluid Flow: by Deville, Fischer and Mund, Cambridge University Press, 499 pp. , 2003 .

[13]  P. Fischer,et al.  High-Order Methods for Incompressible Fluid Flow , 2002 .

[14]  L. Demkowicz.  One and two dimensional elliptic and Maxwell problems , 2006 .

[15]  Nicholas J. Higham.  Stability of a Method for Multiplying Complex Matrices with Three Real Matrix Multiplications , 1992, SIAM J. Matrix Anal. Appl..

[16]  G. Karniadakis,et al.  Spectral/hp Element Methods for CFD , 1999 .