Fast Monte-Carlo algorithms for approximate matrix multiplication

Given an m ? n matrix A and an n ? p matrix B, we present 2 simple and intuitive algorithms to compute an approximation P to the product A ? B, with provable bounds for the norm of the "error matrix" P - A ? B. Both algorithms run in 0(mp+mn+np) time. In both algorithms, we randomly pick s = 0(1) columns of A to form an m ? s matrix S and the corresponding rows of B to form an s ? p matrix R. After scaling the columns of S and the rows of R, we multiply them together to obtain our approximation P. The choice of the probability distribution we use for picking the columns of A and the scaling are the crucial features which enable us to fairly elementary proofs of the error bounds. Our first algorithm can be implemented without storing the matrices A and B in Random Access Memory, provided we can make two passes through the matrices (stored in external memory). The second algorithm has a smaller bound on the 2-norm of the error matrix, but requires storage of A and B in RAM. We also present a fast algorithm that "describes" P as a sum of rank one matrices if B = AT.

[1]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[2]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[3]  V. Strassen.  Gaussian elimination is not optimal , 1969 .

[4]  János Komlós,et al.  The eigenvalues of random symmetric matrices , 1981, Comb..

[5]  Gene H. Golub,et al.  Matrix computations , 1983 .

[6]  Don Coppersmith,et al.  Matrix multiplication via arithmetic progressions , 1987, STOC.

[7]  Susan T. Dumais,et al.  Using Linear Algebra for Intelligent Information Retrieval , 1995, SIAM Rev..

[8]  Edith Cohen,et al.  Approximating matrix multiplication for pattern recognition tasks , 1997, SODA '97.

[9]  Alan M. Frieze,et al.  Fast Monte-Carlo algorithms for finding low-rank approximations , 1998, Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280).

[10]  S. Vempala,et al.  E cient Singular Value Decompositionvia Improved Document SamplingFan , 1999 .

[11]  Alan M. Frieze,et al.  Clustering in large graphs and matrices , 1999, SODA '99.

[12]  Michael Krivelevich,et al.  Approximating the Independence Number and the Chromatic Number in Expected Polynominal Time , 2000, ICALP.

[13]  N. Alon,et al.  On the concentration of eigenvalues of random symmetric matrices , 2000, math-ph/0009032.

[14]  Dimitris Achlioptas,et al.  Fast computation of low rank matrix approximations , 2001, STOC '01.

[15]  V. Vu,et al.  Approximating the Independence Number and the Chromatic Number in Expected Polynomial Time , 2000, J. Comb. Optim..