Low Rank Approximation of a Matrix at Sub-linear Cost

Low rank approximation of a matrix (LRA}) is a highly important area of Numerical Linear and Multilinear Algebra and Data Mining and Analysis with numerous important applications to modern computations. One can operate with LRA of a matrix at sublinear cost, that is, by using much fewer memory cells and flops than the matrix has entries, but no sublinear cost algorithm can compute accurate LRA of the worst case input matrices or even of the matrices of small families of low rank matrices in our Appendix. Nevertheless we prove that some old and new sublinear cost algorithms solve the dual LRA problem, that is, with a high probability compute close LRA of a random matrix admitting LRA. Our tests are in good accordance with our formal study, and we have extended our progress into various directions, in particular to dual Linear Least Squares Regression at sublinear cost.

[1]  C. Pan.  On the existence and computation of rank-revealing LU factorizations , 2000 .

[2]  Petros Drineas,et al.  CUR matrix decompositions for improved data analysis , 2009, Proceedings of the National Academy of Sciences.

[3]  Victor Y. Pan,et al.  Randomized Approximation of Linear Least Squares Regression at Sub-linear Cost , 2019, ArXiv.

[4]  CichockiAndrzej,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization , 2016 .

[5]  S. Goreinov,et al.  The maximum-volume concept in approximation by low-rank matrices , 2001 .

[6]  Mario Bebendorf,et al.  Wideband nested cross approximation for Helmholtz problems , 2015, Numerische Mathematik.

[7]  Alan Edelman,et al.  Tails of Condition Number Distributions , 2005, SIAM J. Matrix Anal. Appl..

[8]  S. Goreinov,et al.  A Theory of Pseudoskeleton Approximations , 1997 .

[9]  S. Goreinov,et al.  Pseudo-skeleton approximations by matrices of maximal volume , 1997 .

[10]  Victor Y. Pan,et al.  New Studies of Randomized Augmentation and Additive Preprocessing , 2014, 1412.5864.

[11]  Ming Gu,et al.  Efficient Algorithms for Computing a Strong Rank-Revealing QR Factorization , 1996, SIAM J. Sci. Comput..

[12]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[13]  G. Golub,et al.  Linear least squares solutions by householder transformations , 1965 .

[14]  Victor Y. Pan,et al.  Low Rank Approximation Directed by Leverage Scores and Computed at Sub-linear Cost , 2019, ArXiv.

[15]  David P. Woodruff,et al.  Sublinear Time Low-Rank Approximation of Distance Matrices , 2018, NeurIPS.

[16]  Victor Y. Pan,et al.  Superfast CUR Approximation of Low Rank Matrices ∗ , 2017 .

[17]  E. Tyrtyshnikov,et al.  TT-cross approximation for multidimensional arrays , 2010 .

[18]  A. Levine,et al.  New estimates of the storage permanence and ocean co-benefits of enhanced rock weathering , 2023, PNAS nexus.

[19]  Sjsu ScholarWorks,et al.  Rank revealing QR factorizations , 2014 .

[20]  Gene H. Golub,et al.  Numerical methods for solving linear least squares problems , 1965, Milestones in Matrix Computation.

[21]  Per Christian Hansen,et al.  Some Applications of the Rank Revealing QR Factorization , 1992, SIAM J. Sci. Comput..

[22]  C. Pan,et al.  Rank-Revealing QR Factorizations and the Singular Value Decomposition , 1992 .

[23]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[24]  Rusins Freivalds,et al.  Fast Probabilistic Algorithms , 1979, MFCS.

[25]  S. Goreinov,et al.  How to find a good submatrix , 2010 .

[26]  Adi Ben-Israel.  A volume associated with m x n matrices , 1992 .

[27]  David P. Woodruff.  Sketching as a Tool for Numerical Linear Algebra , 2014, Found. Trends Theor. Comput. Sci..

[28]  Andrzej Cichocki,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions , 2016, Found. Trends Mach. Learn..

[29]  Alexander Osinsky,et al.  Pseudo-skeleton approximations with better accuracy estimates , 2018 .

[30]  Volkan Cevher,et al.  Practical Sketching Algorithms for Low-Rank Matrix Approximation , 2016, SIAM J. Matrix Anal. Appl..

[31]  Per Christian Hansen,et al.  Computing Truncated Singular Value Decomposition Least Squares Solutions by Rank Revealing QR-Factorizations , 1990, SIAM J. Sci. Comput..

[32]  D. E. Knuth.  Semi-optimal bases for linear dependencies , 1985 .

[33]  V. Pan.  Structured Matrices and Polynomials: Unified Superfast Algorithms , 2001 .

[34]  Victor Pan,et al.  Numerically Safe Gaussian Elimination with No Pivoting , 2015, 1501.05385.

[35]  Christos Boutsidis,et al.  Optimal CUR matrix decompositions , 2014, STOC.

[36]  David P. Woodruff,et al.  Numerical linear algebra in the streaming model , 2009, STOC '09.

[37]  Boris N. Khoromskij,et al.  Efficient Computation of Highly Oscillatory Integrals by Using QTT Tensor Approximation , 2014, Comput. Methods Appl. Math..

[38]  Zizhong Chen,et al.  Condition Numbers of Gaussian Random Matrices , 2005, SIAM J. Matrix Anal. Appl..

[39]  T. Chan.  Rank revealing QR factorizations , 1987 .

[40]  N. Zamarashkin,et al.  New accuracy estimates for pseudoskeleton approximations of matrices , 2016 .

[41]  G. W. Stewart,et al.  Matrix algorithms , 1998 .

[42]  Alexander Osinsky,et al.  Rectangular maximum volume and projective volume search algorithms , 2018, 1809.02334.

[43]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[44]  John Svadlenka,et al.  Sublinear Cost Low Rank Approximation Directed by Leverage Scores , 2019 .

[45]  E. Tyrtyshnikov.  Mosaic-Skeleton approximations , 1996 .

[46]  Victor Y. Pan,et al.  Low Rank Approximation at Sublinear Cost by Means of Subspace Sampling , 2019, ArXiv.

[47]  Joel A. Tropp,et al.  Improved Analysis of the subsampled Randomized Hadamard Transform , 2010, Adv. Data Sci. Adapt. Anal..

[48]  Victor Y. Pan,et al.  Random multipliers numerically stabilize Gaussian and block Gaussian elimination: Proofs and an extension to low-rank approximation ☆ , 2014, 1406.5802.

[49]  G. Stewart.  Error and Perturbation Bounds for Subspaces Associated with Certain Eigenvalue Problems , 1973 .

[50]  A. I. Osinsky,et al.  On the Existence of a Nearly Optimal Skeleton Approximation of a Matrix in the Frobenius Norm , 2018 .

[51]  A. Edelman.  Eigenvalues and condition numbers of random matrices , 1988 .

[52]  David P. Woodruff,et al.  Low rank approximation with entrywise l1-norm error , 2017, STOC.

[53]  Shang-Hua Teng,et al.  Smoothed Analysis of the Condition Numbers and Growth Factors of Matrices , 2003, SIAM J. Matrix Anal. Appl..

[54]  Alan C. Elliott,et al.  Statistical Analysis Quick Reference Guidebook: With SPSS Examples , 2006 .

[55]  Eugene E. Tyrtyshnikov,et al.  Quasioptimality of skeleton approximation of a matrix in the Chebyshev norm , 2011 .

[56]  Mario Bebendorf,et al.  Approximation of boundary element matrices , 2000, Numerische Mathematik.

[57]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[58]  V. Rokhlin,et al.  A fast randomized algorithm for the approximation of matrices ✩ , 2007 .

[59]  Peter Lancaster,et al.  The theory of matrices , 1969 .

[60]  N. Kishore Kumar,et al.  Literature survey on low rank approximation of matrices , 2016, ArXiv.

[61]  Victor Y. Pan,et al.  Primitive and Cynical Low Rank Approximation, Preprocessing and Extensions , 2016 .

[62]  Yudong Chen,et al.  Coherent Matrix Completion , 2013, ICML.

[63]  James Demmel,et al.  Error bounds from extra-precise iterative refinement , 2006, TOMS.

[64]  J. Demmel.  The Probability That a Numerical, Analysis Problem Is Difficult , 2013 .

[65]  Victor Y. Pan.  Some Recent and New Techniques for Superfast (Sublinear Cost) Low Rank Approximation , 2018 .

[66]  M. Bebendorf,et al.  Accelerating Galerkin BEM for linear elasticity using adaptive cross approximation , 2006 .

[67]  Å. Björck.  Numerical Methods in Matrix Computations , 2014 .

[68]  Eugene E. Tyrtyshnikov,et al.  Incomplete Cross Approximation in the Mosaic-Skeleton Method , 2000, Computing.

[69]  Mario Bebendorf,et al.  Adaptive Cross Approximation of Multivariate Functions , 2011 .

[70]  Michael W. Mahoney.  Randomized Algorithms for Matrices and Data , 2011, Found. Trends Mach. Learn..

[71]  Victor Y. Pan,et al.  Refinement of Low Rank Approximation of a Matrix at Sub-linear Cost. , 2019 .

[72]  Sergej Rjasanow,et al.  Adaptive Low-Rank Approximation of Collocation Matrices , 2003, Computing.

[73]  V. Rokhlin,et al.  A randomized algorithm for the approximation of matrices , 2006 .

[74]  Victor Y. Pan,et al.  N A ] 5 J un 2 01 6 Fast Low-rank Approximation of a Matrix : Novel Insights , Novel Multipliers , and Extensions ∗ , 2016 .

[75]  T. Hwang,et al.  Rank revealing LU factorizations , 1992 .

[76]  S. Muthukrishnan,et al.  Relative-Error CUR Matrix Decompositions , 2007, SIAM J. Matrix Anal. Appl..