Randomized Subspace Iteration: Analysis of Canonical Angles and Unitarily Invariant Norms

This paper is concerned with the analysis of the randomized subspace iteration for the computation of low-rank approximations. We present three different kinds of bounds. First, we derive both bounds for the canonical angles between the exact and the approximate singular subspaces. Second, we derive bounds for the low-rank approximation in any unitarily invariant norm (including the Schatten-p norm). This generalizes the bounds for Spectral and Frobenius norms found in the literature. Third, we present bounds for the accuracy of the singular values. The bounds are structural in that they are applicable to any starting guess, be it random or deterministic, that satisfies some minimal assumptions. Specialized bounds are provided when a Gaussian random matrix is used as the starting guess. Numerical experiments demonstrate the effectiveness of the proposed bounds.

[1]  Yuji Nakatsukasa,et al.  Accuracy of singular vectors obtained by projection-based SVD methods , 2017 .

[2]  Ilse C. F. Ipsen,et al.  Low-Rank Matrix Approximations Do Not Need a Singular Value Gap , 2018, SIAM J. Matrix Anal. Appl..

[3]  Danny C. Sorensen,et al.  A DEIM Induced CUR Factorization , 2014, SIAM J. Sci. Comput..

[4]  Shang-Hua Teng,et al.  Smoothed Analysis of the Condition Numbers and Growth Factors of Matrices , 2003, SIAM J. Matrix Anal. Appl..

[5]  Eun-Young Lee.  Extension of Rotfel’d Theorem , 2011 .

[6]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[7]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[8]  Michael W. Mahoney.  Randomized Algorithms for Matrices and Data , 2011, Found. Trends Mach. Learn..

[9]  Gene H. Golub,et al.  Numerical methods for computing angles between linear subspaces , 1971, Milestones in Matrix Computation.

[10]  Gene H. Golub,et al.  Matrix computations , 1983 .

[11]  D. Sorensen.  Numerical methods for large eigenvalue problems , 2002, Acta Numerica.

[12]  Christos Boutsidis,et al.  Spectral Clustering via the Power Method - Provably , 2013, ICML.

[13]  Michiel E. Hochstenbach,et al.  Harmonic and Refined Extraction Methods for the Singular Value Problem, with Applications in Least Squares Problems , 2004 .

[14]  Steven L. Brunton,et al.  Randomized Dynamic Mode Decomposition , 2017, SIAM J. Appl. Dyn. Syst..

[15]  Misha Elena Kilmer,et al.  A randomized tensor singular value decomposition based on the t‐product , 2016, Numer. Linear Algebra Appl..

[16]  Siam J. Sci,et al.  SUBSPACE ITERATION RANDOMIZATION AND SINGULAR VALUE PROBLEMS , 2015 .

[17]  Ping Tak Peter Tang,et al.  FEAST As A Subspace Iteration Eigensolver Accelerated By Approximate Spectral Projection , 2013, SIAM J. Matrix Anal. Appl..

[18]  Lek-Heng Lim,et al.  Schubert Varieties and Distances between Subspaces of Different Dimensions , 2014, SIAM J. Matrix Anal. Appl..

[19]  Anthony Nouy,et al.  Randomized linear algebra for model reduction. Part I: Galerkin methods and error estimation , 2018, Advances in Computational Mathematics.

[20]  Ilse C. F. Ipsen,et al.  Conditioning of Leverage Scores and Computation by QR Decomposition , 2015, SIAM J. Matrix Anal. Appl..

[21]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[22]  Christos Boutsidis,et al.  Efficient Dimensionality Reduction for Canonical Correlation Analysis , 2012, SIAM J. Sci. Comput..

[23]  Volkan Cevher,et al.  Practical Sketching Algorithms for Low-Rank Matrix Approximation , 2016, SIAM J. Matrix Anal. Appl..

[24]  F. L. Bauer.  Das Verfahren der Treppeniteration und verwandte Verfahren zur Lösung algebraischer Eigenwertprobleme , 1957 .

[25]  Ilse C. F. Ipsen,et al.  Structural Convergence Results for Approximation of Dominant Subspaces from Block Krylov Spaces , 2016, SIAM J. Matrix Anal. Appl..

[26]  P. Wedin.  On angles between subspaces of a finite dimensional inner product space , 1983 .