Optimization techniques for small matrix multiplication

The complexity of matrix multiplication has attracted a lot of attention in the last forty years. In this paper, instead of considering asymptotic aspects of this problem, we are interested in reducing the cost of multiplication for matrices of small size, say up to 30. Following the previous work of Probert & Fischer, Smith, and Mezzarobba, in a similar vein, we base our approach on the previous algorithms for small matrices, due to Strassen, Winograd, Pan, Laderman, and others and show how to exploit these standard algorithms in an improved way. We illustrate the use of our results by generating multiplication codes over various rings, such as integers, polynomials, differential operators and linear recurrence operators.

[1]  V. Strassen.  Gaussian elimination is not optimal , 1969 .

[2]  Yuefan Deng,et al.  New trends in high performance computing , 2001, Parallel Computing.

[3]  D. V. Chudnovsky,et al.  Computers in Mathematics. , 1991 .

[4]  B. Beckermann,et al.  A Uniform Approach for the Fast Computation of Matrix-Type Padé Approximants , 1994, SIAM J. Matrix Anal. Appl..

[5]  John J. Cannon,et al.  The Magma Algebra System I: The User Language , 1997, J. Symb. Comput..

[6]  O. M. Makarov.  An algorithm for multiplying 3 x 3 matrics , 1987 .

[7]  John E. Hopcroft,et al.  Duality applied to the complexity of matrix multiplications and other bilinear forms , 1973, STOC '73.

[8]  G. Schachtel,et al.  A Noncommutative Algorithm for Multiplying 5*5 Matrices Using 103 Multiplications , 1978, Inf. Process. Lett..

[9]  R. Brent.  Algorithms for matrix multiplication , 1970 .

[10]  V. Pan.  How can we speed up matrix multiplication , 1984 .

[11]  Markus Bläser,et al.  On the complexity of the multiplication of matrices of small formats , 2003, J. Complex..

[12]  Nicolas Le Roux,et al.  Products of ordinary differential operators by evaluation and interpolation , 2008, ISSAC '08.

[13]  Markus Bläser.  Beyond the Alder-Strassen bound , 2005, Theor. Comput. Sci..

[14]  Abraham Waksman.  On Winograd's Algorithm for Inner Products , 1970, IEEE Transactions on Computers.

[15]  Ondrej Sýkora.  A Fast Non-Commutative Algorithm for Matrix Multiplication , 1977, MFCS.

[16]  David Y. Y. Yun,et al.  Fast Solution of Toeplitz Systems of Equations and Computation of Padé Approximants , 1980, J. Algorithms.

[17]  Arnold Schönhage,et al.  Partial and Total Matrix Multiplication , 1981, SIAM J. Comput..

[18]  V. Strassen.  The asymptotic spectrum of tensors. , 1988 .

[19]  L. R. Kerr,et al.  On Minimizing the Number of Multiplications Necessary for Matrix Multiplication , 1969 .

[20]  Don Coppersmith,et al.  Matrix multiplication via arithmetic progressions , 1987, STOC.

[21]  Alexandru Nicolau,et al.  Adaptive Strassen's matrix multiplication , 2007, ICS '07.

[22]  Rodney W. Johnson,et al.  Noncommutative Bilinear Algorithms for 3 x 3 Matrix Multiplication , 1986, SIAM J. Comput..

[23]  Joris van der Hoeven.  FFT-like Multiplication of Linear Differential Operators , 2002, J. Symb. Comput..

[24]  Shmuel Winograd,et al.  A New Algorithm for Inner Product , 1968, IEEE Transactions on Computers.

[25]  Victor Y. Pan,et al.  New Fast Algorithms for Matrix Operations , 1980, SIAM J. Comput..

[26]  V. Pan,et al.  Trilinear aggregating with implicit canceling for a new acceleration of matrix multiplication , 1982 .

[27]  Julian D. Laderman,et al.  A noncommutative algorithm for multiplying $3 \times 3$ matrices using 23 multiplications , 1976 .

[28]  Julian D. Laderman,et al.  On practical algorithms for accelerated matrix multiplication , 1992 .

[29]  Éric Schost,et al.  Complexity results for triangular sets , 2003, J. Symb. Comput..

[30]  O. M. Makarov.  A non-commutative algorithm for multiplying 5×5 matrices using one hundred multiplications , 1988 .

[31]  Shmuel Winograd,et al.  On multiplication of 2 × 2 matrices , 1971 .

[32]  Igor E. Kaporin,et al.  The aggregation and cancellation techniques as a practical tool for faster matrix multiplication , 2004, Theor. Comput. Sci..