Perturbations of CUR Decompositions

The CUR decomposition is a factorization of a low-rank matrix obtained by selecting certain column and row submatrices of it. We perform a thorough investigation of what happens to such decompositions in the presence of noise. Since CUR decompositions are non-uniquely formed, we investigate several variants and give perturbation estimates for each in terms of the magnitude of the noise matrix in a broad class of norms which includes all Schatten $p$--norms. The estimates given here are qualitative and illustrate how the choice of columns and rows affects the quality of the approximation, and additionally we obtain new state-of-the-art bounds for some variants of CUR approximations.

[1]  Ivan V. Oseledets,et al.  Rectangular maximum-volume submatrices and their applications , 2015, ArXiv.

[2]  Christos Boutsidis,et al.  Optimal CUR matrix decompositions , 2014, STOC.

[3]  Ilse C. F. Ipsen,et al.  Low-Rank Matrix Approximations Do Not Need a Singular Value Gap , 2018, SIAM J. Matrix Anal. Appl..

[4]  Volkan Cevher,et al.  Streaming Low-Rank Matrix Approximation with an Application to Scientific Simulation , 2019, SIAM J. Sci. Comput..

[5]  G. W. Stewart,et al.  Four algorithms for the the efficient computation of truncated pivoted QR approximations to a sparse matrix , 1999, Numerische Mathematik.

[6]  Danny C. Sorensen,et al.  A DEIM Induced CUR Factorization , 2014, SIAM J. Sci. Comput..

[7]  Victor Y. Pan,et al.  Low Rank Approximation of a Matrix at Sub-linear Cost , 2019, ArXiv.

[8]  Petros Drineas,et al.  CUR matrix decompositions for improved data analysis , 2009, Proceedings of the National Academy of Sciences.

[9]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[10]  E. Candès,et al.  Compressed sensing and robust recovery of low rank matrices , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[11]  Laurent Demanet,et al.  Sublinear Randomized Algorithms for Skeleton Decompositions , 2011, SIAM J. Matrix Anal. Appl..

[12]  G. Stewart.  On the Perturbation of Pseudo-Inverses, Projections and Linear Least Squares Problems , 1977 .

[13]  Alexander Osinsky,et al.  Rectangular maximum volume and projective volume search algorithms , 2018, 1809.02334.

[14]  L. Mirsky.  SYMMETRIC GAUGE FUNCTIONS AND UNITARILY INVARIANT NORMS , 1960 .

[15]  Gene H. Golub,et al.  Matrix computations , 1983 .

[16]  Petros Drineas,et al.  On the Nyström Method for Approximating a Gram Matrix for Improved Kernel-Based Learning , 2005, J. Mach. Learn. Res..

[17]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[18]  Roberto Tron RenVidal.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007 .

[19]  Michael W. Mahoney,et al.  Revisiting the Nystrom Method for Improved Large-scale Machine Learning , 2013, J. Mach. Learn. Res..

[20]  S. Goreinov,et al.  A Theory of Pseudoskeleton Approximations , 1997 .

[21]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Severnyi Kavkaz.  Pseudo-Skeleton Approximations by Matrices of Maximal Volume , 2022 .

[23]  S. Goreinov,et al.  Pseudo-skeleton approximations by matrices of maximal volume , 1997 .

[24]  Petros Drineas,et al.  FAST MONTE CARLO ALGORITHMS FOR MATRICES III: COMPUTING A COMPRESSED APPROXIMATE MATRIX DECOMPOSITION∗ , 2004 .

[25]  Volkan Cevher,et al.  Fixed-Rank Approximation of a Positive-Semidefinite Matrix from Streaming Data , 2017, NIPS.

[26]  G. Stewart.  Perturbation theory for the singular value decomposition , 1990 .

[27]  Shusen Wang,et al.  Scalable Kernel K-Means Clustering with Nystrom Approximation: Relative-Error Bounds , 2017, J. Mach. Learn. Res..

[28]  Madeleine Udell,et al.  Why Are Big Data Matrices Approximately Low Rank? , 2017, SIAM J. Math. Data Sci..

[29]  Per-Gunnar Martinsson,et al.  Efficient algorithms for cur and interpolative matrix decompositions , 2014, Adv. Comput. Math..

[30]  S. Goreinov,et al.  How to find a good submatrix , 2010 .

[31]  Alexander Osinsky,et al.  Pseudo-skeleton approximations with better accuracy estimates , 2018 .

[32]  Aarti Singh,et al.  Provably Correct Algorithms for Matrix Column Subset Selection with Selectively Sampled Data , 2015, J. Mach. Learn. Res..

[33]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[34]  S. Muthukrishnan,et al.  Relative-Error CUR Matrix Decompositions , 2007, SIAM J. Matrix Anal. Appl..

[35]  Stephen Becker,et al.  Improved Fixed-Rank Nyström Approximation via QR Decomposition: Practical and Theoretical Aspects , 2017, Neurocomputing.

[36]  Akram Aldroubi,et al.  CUR Decompositions, Similarity Matrices, and Subspace Clustering , 2017, Front. Appl. Math. Stat..

[37]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..