Minimizing Communication in Linear Algebra

Algorithms have two kinds of costs: arithmetic and communication, by which we mean moving data either between levels of a memory hierarchy (in the sequential case) or between processors over a network (in the parallel case). Communication costs can already exceed arithmetic costs by orders of magnitude, and the gap is growing exponentially over time, so our goal is to design linear algebra algorithms that minimize communication. First, we show how to extend known communication lower bounds for O(n) dense matrix multiplication to all direct linear algebra, i.e. for solving linear systems, least squares problems, eigenproblems and the SVD, for dense or sparse matrices, and for sequential or parallel machines. We also describe new algorithms that attain these lower bounds; some implementations attain large speed ups over conventional algorithms. Second, we show how to minimize communication in Krylov-subspace methods for solving sparse linear system and eigenproblems, and again demonstrate new algorithms with significant speedups. Monday December 13 2010 4:30 PM Building 2, Room 105 Refreshments are available in Building 2, Room 290 (Math Common Room) between 3:30 – 4:30 PM Applied Math Colloquium: http://www-math.mit.edu/amc/fall10 Mathematics Department: http://www-math.mit.edu To sign up for Applied Mathematics Colloquium announcements, please contact avisha@math.mit.edu Massachusetts Institute of Technology Department of Mathematics Cambridge, MA 02139