Graph Expansion Analysis for Communication Costs of Fast Rectangular Matrix Multiplication

Graph expansion analysis of computational DAGs is useful for obtaining communication cost lower bounds where previous methods, such as geometric embedding, are not applicable. This has recently been demonstrated for Strassen's and Strassen-like fast square matrix multiplication algorithms. Here we extend the expansion analysis approach to fast algorithms for rectangular matrix multiplication, obtaining a new class of communication cost lower bounds. These apply, for example to the algorithms of Bini et al. (1979) and the algorithms of Hopcroft and Kerr (1971). Some of our bounds are proved to be optimal.

[1]  Dario Bini.  Relations between exact and approximate bilinear algorithms. Applications , 1980 .

[2]  Don Coppersmith,et al.  Rectangular Matrix Multiplication Revisited , 1997, J. Complex..

[3]  Patrick C. Fischer,et al.  Efficient Procedures for Using Matrix Algorithms , 1974, ICALP.

[4]  D. Coppersmiths.  RAPID MULTIPLICATION OF RECTANGULAR MATRICES * , 2014 .

[5]  J. Spinrad,et al.  Between O(nm) and O(nα) , 2003, SODA '03.

[6]  James Demmel,et al.  Minimizing Communication in Numerical Linear Algebra , 2009, SIAM J. Matrix Anal. Appl..

[7]  John E. Savage.  Space-Time Tradeoffs in Memory Hierarchies , 1994 .

[8]  W. Marsden.  I and J , 2012 .

[9]  Haim Kaplan,et al.  Colored intersection searching via sparse rectangular matrix multiplication , 2006, SCG '06.

[10]  Andrea Pietracaprina,et al.  On the Space and Access Complexity of Computation DAGs , 2000, WG.

[11]  Milena Mihail,et al.  Conductance and convergence of Markov chains-a combinatorial treatment of expanders , 1989, 30th Annual Symposium on Foundations of Computer Science.

[12]  Uri Zwick,et al.  All pairs shortest paths using bridging sets and rectangular matrix multiplication , 2000, JACM.

[13]  P. Knight.  Fast rectangular matrix multiplication and QR decomposition , 1995 .

[14]  Dror Irony,et al.  Communication lower bounds for distributed-memory matrix multiplication , 2004, J. Parallel Distributed Comput..

[15]  Shmuel Winograd,et al.  On multiplication of 2 × 2 matrices , 1971 .

[16]  James Demmel,et al.  Communication-optimal parallel algorithm for strassen's matrix multiplication , 2012, SPAA '12.

[17]  John E. Hopcroft,et al.  Duality applied to the complexity of matrix multiplications and other bilinear forms , 1973, STOC '73.

[18]  James Demmel,et al.  Communication-Avoiding Parallel Strassen: Implementation and performance , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[19]  Noga Alon,et al.  An elementary construction of constant-degree expanders , 2007, SODA '07.

[20]  James Demmel,et al.  Brief announcement: strong scaling of matrix multiplication algorithms and memory-independent communication lower bounds , 2012, SPAA '12.

[21]  F. P. Preparata,et al.  Processor—Time Tradeoffs under Bounded-Speed Message Propagation: Part I, Upper Bounds , 1995, Theory of Computing Systems.

[22]  Michael Clausen,et al.  Algebraic complexity theory , 1997, Grundlehren der mathematischen Wissenschaften.

[23]  Victor Y. Pan,et al.  Fast Rectangular Matrix Multiplication and Applications , 1998, J. Complex..

[24]  Leslie G. Valiant,et al.  Size Bounds for Superconcentrators , 1983, Theor. Comput. Sci..

[25]  Peter A. Beling,et al.  Using Fast Matrix Multiplication to Find Basic Solutions , 1998, Theoretical Computer Science.

[26]  Robin Milner,et al.  On Observing Nondeterminism and Concurrency , 1980, ICALP.

[27]  H. T. Kung,et al.  I/O complexity: The red-blue pebble game , 1981, STOC '81.

[28]  V. Pan,et al.  Fast rectangular matrix multiplication and some applications , 2008 .

[29]  Grazia Lotti,et al.  O(n2.7799) Complexity for n*n Approximate Matrix Multiplication , 1979, Inf. Process. Lett..

[30]  Victor Y. Pan,et al.  Parallel Evaluation of the Determinant and of the Inverse of a Matrix , 1989, Inf. Process. Lett..

[31]  Victor Y. Pan,et al.  Fast rectangular matrix multiplications and improving parallel matrix computations , 1997, PASCO '97.

[32]  Avi Wigderson,et al.  Entropy waves, the zig-zag graph product, and new constant-degree expanders and extractors , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[33]  Franco P. Preparata,et al.  Processor—Time Tradeoffs under Bounded-Speed Message Propagation: Part II, Lower Bounds , 1999, Theory of Computing Systems.

[34]  Grazia Lotti,et al.  On the Asymptotic Complexity of Rectangular Matrix Multiplication , 1983, Theor. Comput. Sci..

[35]  V. Strassen.  Gaussian elimination is not optimal , 1969 .

[36]  Raphael Yuster,et al.  Detecting short directed cycles using rectangular matrix multiplication and dynamic programming , 2004, SODA '04.

[37]  Raphael Yuster,et al.  Fast sparse matrix multiplication , 2004, TALG.

[38]  L. R. Kerr,et al.  On Minimizing the Number of Multiplications Necessary for Matrix Multiplication , 1969 .