An improved analysis and unified perspective on deterministic and randomized low rank matrix approximations

We introduce a Generalized LU-Factorization (\textbf{GLU}) for low-rank matrix approximation. We relate this to past approaches and extensively analyze its approximation properties. The established deterministic guarantees are combined with sketching ensembles satisfying Johnson-Lindenstrauss properties to present complete bounds. Particularly good performance is shown for the sub-sampled randomized Hadamard transform (SRHT) ensemble. Moreover, the factorization is shown to unify and generalize many past algorithms. It also helps to explain the effect of sketching on the growth factor during Gaussian Elimination.

[1]  David P. Woodruff,et al.  Low rank approximation and regression in input sparsity time , 2012, STOC '13.

[2]  Tamás Sarlós,et al.  Improved Approximation Algorithms for Large Matrices via Random Projections , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[3]  Zizhong Chen,et al.  Condition Numbers of Gaussian Random Matrices , 2005, SIAM J. Matrix Anal. Appl..

[4]  James Demmel,et al.  Low Rank Approximation of a Sparse Matrix Based on LU Factorization with Column and Row Tournament Pivoting , 2018, SIAM J. Sci. Comput..

[5]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[6]  Ming Gu,et al.  Subspace Iteration Randomization and Singular Value Problems , 2014, SIAM J. Sci. Comput..

[7]  Huy L. Nguyen,et al.  OSNAP: Faster Numerical Linear Algebra Algorithms via Sparser Subspace Embeddings , 2012, 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.

[8]  M. Gu,et al.  Strong rank revealing LU factorizations , 2003 .

[9]  David P. Woodru.  Sketching as a Tool for Numerical Linear Algebra , 2014 .

[10]  Christos Boutsidis,et al.  Improved Matrix Algorithms via the Subsampled Randomized Hadamard Transform , 2012, SIAM J. Matrix Anal. Appl..

[11]  J. Bourgain,et al.  Toward a unified theory of sparse dimensionality reduction in Euclidean space , 2015 .

[12]  James Demmel,et al.  Fast linear algebra is stable , 2006, Numerische Mathematik.

[13]  Petros Drineas,et al.  CUR matrix decompositions for improved data analysis , 2009, Proceedings of the National Academy of Sciences.

[14]  Xiaoye S. Li,et al.  Using Random Butterfly Transformations to Avoid Pivoting in Sparse Direct Methods , 2014, VECPAR.

[15]  Ioana Dumitriu.  Smallest eigenvalue distributions for two classes of $\beta$-Jacobi ensembles , 2010 .

[16]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[17]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[18]  Ming Gu,et al.  Efficient Algorithms for Computing a Strong Rank-Revealing QR Factorization , 1996, SIAM J. Sci. Comput..

[19]  Cameron Musco,et al.  Randomized Block Krylov Methods for Stronger and Faster Approximate Singular Value Decomposition , 2015, NIPS.

[20]  David P. Woodruff.  Sketching as a Tool for Numerical Linear Algebra , 2014, Found. Trends Theor. Comput. Sci..

[21]  J. Demmel,et al.  An inverse free parallel spectral divide and conquer algorithm for nonsymmetric eigenproblems , 1997 .

[22]  Tiefeng Jiang,et al.  Maxima of entries of Haar distributed matrices , 2005 .