Accuracy of singular vectors obtained by projection-based SVD methods

The standard approach to computing an approximate SVD of a large-scale matrix is to project it onto lower-dimensional trial subspaces from both sides, compute the SVD of the small projected matrix, and project it back to the original space. This results in a low-rank approximate SVD to the original matrix, and we can then obtain approximate left and right singular subspaces by extracting subsets from the approximate SVD. In this work we assess the quality of the extraction process in terms of the accuracy of the approximate singular subspaces, measured by the angle between the exact and extracted subspaces (relative to the angle between the exact and trial subspaces). The main message is that the extracted approximate subspaces are optimal usually to within a modest constant.

[1]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[2]  Charles R. Johnson,et al.  Matrix Analysis, 2nd Ed , 2012 .

[3]  Andrew Knyazev,et al.  New estimates for Ritz vectors , 1997, Math. Comput..

[4]  Chandler Davis.  The rotation of eigenvectors by a perturbation , 1963 .

[5]  Ren-Cang Li,et al.  A note on eigenvalues of perturbed Hermitian matrices , 2005 .

[6]  W. Kahan,et al.  The Rotation of Eigenvectors by a Perturbation. III , 1970 .

[7]  Roy Mathias,et al.  Quadratic Residual Bounds for the Hermitian Eigenvalue Problem , 1998 .

[8]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[9]  M. Saunders,et al.  Towards a Generalized Singular Value Decomposition , 1981 .

[10]  D. Sorensen.  Numerical methods for large eigenvalue problems , 2002, Acta Numerica.

[11]  Michiel E. Hochstenbach,et al.  Harmonic and Refined Extraction Methods for the Singular Value Problem, with Applications in Least Squares Problems , 2004 .

[12]  Jack Dongarra,et al.  Templates for the Solution of Algebraic Eigenvalue Problems , 2000, Software, environments, tools.

[13]  Christopher A. Beattie,et al.  Galerkin eigenvector approximations , 1998, Math. Comput..

[14]  Siam J. Sci,et al.  SUBSPACE ITERATION RANDOMIZATION AND SINGULAR VALUE PROBLEMS , 2015 .

[15]  R. Morgan.  Computing Interior Eigenvalues of Large Matrices , 1991 .

[16]  G. Stewart,et al.  Matrix Perturbation Theory , 1990 .

[17]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[18]  Gene H. Golub,et al.  Matrix computations , 1983 .

[19]  G. Stewart.  A generalization of Saad's theorem on Rayleigh–Ritz approximations , 2001 .

[20]  G. Stewart.  Matrix Algorithms, Volume II: Eigensystems , 2001 .

[21]  P. Wedin.  Perturbation bounds in connection with singular value decomposition , 1972 .

[22]  Y. Saad.  Numerical Methods for Large Eigenvalue Problems , 2011 .

[23]  Ren-Cang Li,et al.  Convergence of the block Lanczos method for eigenvalue clusters , 2014, Numerische Mathematik.

[24]  Michiel E. Hochstenbach,et al.  A Jacobi-Davidson Type SVD Method , 2001, SIAM J. Sci. Comput..

[25]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[26]  Roland W. Freund,et al.  Algorithms and perturbation theory for matrix eigenvalue problems and the singular value decomposition , 2011 .

[27]  Chandler Davis,et al.  Perturbation of spectral subspaces and solution of linear operator equations , 1983 .