Low Rank Approximation of a Sparse Matrix Based on LU Factorization with Column and Row Tournament Pivoting

In this paper we present an algorithm for computing a low rank approximation of a sparse matrix based on a truncated LU factorization with column and row permutations. We present various approaches for determining the column and row permutations that show a trade-off between speed versus deterministic/probabilistic accuracy. We show that if the permutations are chosen by using tournament pivoting based on QR factorization, then the obtained truncated LU factorization with column/row tournament pivoting, LU_CRTP, satisfies bounds on the singular values which have similarities with the ones obtained by a communication avoiding rank revealing QR factorization. Experiments on challenging matrices show that LU_CRTP provides a good low rank approximation of the input matrix and it is less expensive than the rank revealing QR factorization in terms of computational and memory usage costs, while also minimizing the communication cost. We also compare the computational complexity of our algorithm with randomized a...

[1]  G. Golub,et al.  Linear least squares solutions by householder transformations , 1965 .

[2]  Per Christian Hansen,et al.  Regularization Tools version 4.0 for Matlab 7.3 , 2007, Numerical Algorithms.

[3]  Timothy A. Davis,et al.  Algorithm 915, SuiteSparseQR: Multifrontal multithreaded rank-revealing sparse QR factorization , 2011, TOMS.

[4]  J. Cullum,et al.  Lanczos Algorithms for Large Symmetric Eigenvalue Computations Vol. I Theory , 1984 .

[5]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[6]  J. Pasciak,et al.  Computer solution of large sparse positive definite systems , 1982 .

[7]  S. Goreinov,et al.  A Theory of Pseudoskeleton Approximations , 1997 .

[8]  Philipp Birken,et al.  Numerical Linear Algebra , 2011, Encyclopedia of Parallel Computing.

[9]  James Demmel,et al.  Communication-optimal Parallel and Sequential QR and LU Factorizations , 2008, SIAM J. Sci. Comput..

[10]  A. George.  Nested Dissection of a Regular Finite Element Mesh , 1973 .

[11]  James Demmel,et al.  LU Factorization with Panel Rank Revealing Pivoting and Its Communication Avoiding Version , 2012, SIAM J. Matrix Anal. Appl..

[12]  Christian H. Bischof,et al.  A Parallel QR Factorization Algorithm with Controlled Local Pivoting , 1991, SIAM J. Sci. Comput..

[13]  James Demmel,et al.  CALU: A Communication Optimal LU Factorization Algorithm , 2011, SIAM J. Matrix Anal. Appl..

[14]  J. Cullum,et al.  Lanczos algorithms for large symmetric eigenvalue computations , 1985 .

[15]  R. Tarjan,et al.  A Separator Theorem for Planar Graphs , 1977 .

[16]  Ilse C. F. Ipsen,et al.  On Rank-Revealing Factorisations , 1994, SIAM J. Matrix Anal. Appl..

[17]  H. Walker,et al.  Numerical Linear Algebra with Applications , 1994 .

[18]  L. Trefethen,et al.  Numerical linear algebra , 1997 .

[19]  Jack Dongarra,et al.  LAPACK Users' Guide, 3rd ed. , 1999 .

[20]  Ming Gu,et al.  Efficient Algorithms for Computing a Strong Rank-Revealing QR Factorization , 1996, SIAM J. Sci. Comput..

[21]  A. George,et al.  A data structure for sparse QR and LU factorizations , 1988 .

[22]  M. Gu,et al.  Strong rank revealing LU factorizations , 2003 .

[23]  G. W. Stewart,et al.  Four algorithms for the the efficient computation of truncated pivoted QR approximations to a sparse matrix , 1999, Numerische Mathematik.

[24]  Michael W. Mahoney.  Randomized Algorithms for Matrices and Data , 2011, Found. Trends Mach. Learn..

[25]  D. Sorensen.  Numerical methods for large eigenvalue problems , 2002, Acta Numerica.

[26]  James Demmel,et al.  Block LU factorization , 1992 .

[27]  James Demmel,et al.  Minimizing Communication in Numerical Linear Algebra , 2009, SIAM J. Matrix Anal. Appl..

[28]  Timothy A. Davis,et al.  A column approximate minimum degree ordering algorithm , 2000, TOMS.

[29]  George Karypis,et al.  A Software Package for Partitioning Unstructured Graphs , Partitioning Meshes , and Computing Fill-Reducing Orderings of Sparse Matrices Version 5 . 0 , 1998 .

[30]  C. Pan.  On the existence and computation of rank-revealing LU factorizations , 2000 .

[31]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[32]  E. Ng,et al.  Separators and structure prediction in sparse orthogonal factorization , 1997 .

[33]  G. W. Stewart,et al.  The QLP Approximation to the Singular Value Decomposition , 1999, SIAM J. Sci. Comput..

[34]  Gene H. Golub,et al.  Matrix computations , 1983 .

[35]  Gene H. Golub,et al.  Numerical methods for solving linear least squares problems , 1965, Milestones in Matrix Computation.

[36]  Per Christian Hansen,et al.  Some Applications of the Rank Revealing QR Factorization , 1992, SIAM J. Sci. Comput..

[37]  C. Pan,et al.  Rank-Revealing QR Factorizations and the Singular Value Decomposition , 1992 .

[38]  David P. Woodruff,et al.  Low rank approximation and regression in input sparsity time , 2013, STOC '13.