A Preconditioned Hybrid SVD Method for Accurately Computing Singular Triplets of Large Matrices

The computation of a few singular triplets of large, sparse matrices is a challenging task, especially when the smallest magnitude singular values are needed in high accuracy. Most recent efforts try to address this problem through variations of the Lanczos bidiagonalization method, but they are still challenged even for medium matrix sizes due to the difficulty of the problem. We propose a novel SVD approach that can take advantage of preconditioning and of any well designed eigensolver to compute both largest and smallest singular triplets. Accuracy and efficiency is achieved through a hybrid, two-stage meta-method, PHSVDS. In the first stage, PHSVDS solves the normal equations up to the best achievable accuracy. If further accuracy is required, the method switches automatically to an eigenvalue problem with the augmented matrix. Thus it combines the advantages of the two stages, faster convergence and accuracy, respectively. For the augmented matrix, solving the interior eigenvalue is facilitated by a proper use of the good initial guesses from the first stage and an efficient implementation of the refined projection method. We also discuss how to precondition PHSVDS and to cope with some issues that arise. Numerical experiments illustrate the efficiency and robustness of the method.

[1]  Jennifer A. Scott,et al.  Chebyshev acceleration of iterative refinement , 2014, Numerical Algorithms.

[2]  Lingfei Wu,et al.  Enhancing the PRIMME Eigensolver for Computing Accurately Singular Triplets of Large Matrices , 2014 .

[3]  J. Dongarra Improving the Accuracy of Computed Singular Values , 1983 .

[4]  Andreas Stathopoulos,et al.  PRIMME_SVDS: A Preconditioned SVD Solver for Computing Accurately Singular Triplets of Large Matrices based on the PRIMME Eigensolver , 2014, ArXiv.

[5]  Michele Benzi,et al.  A Robust Preconditioner with Low Memory Requirements for Large Sparse Least Squares Problems , 2003, SIAM J. Sci. Comput..

[6]  Miloud Sadkane,et al.  Computation of the fundamental singular subspace of a large matrix , 1997 .

[7]  Henk A. van der Vorst,et al.  Computational methods for large eigenvalue problems , 2002 .

[8]  Gerard L. G. Sleijpen,et al.  A Jacobi-Davidson Iteration Method for Linear Eigenvalue Problems , 1996, SIAM J. Matrix Anal. Appl..

[9]  Andreas Stathopoulos,et al.  PRIMME: preconditioned iterative multimethod eigensolver—methods and software description , 2010, TOMS.

[10]  Kesheng Wu,et al.  Thick-Restart Lanczos Method for Large Symmetric Eigenvalue Problems , 2000, SIAM J. Matrix Anal. Appl..

[11]  J. Cullum,et al.  A Lanczos Algorithm for Computing Singular Values and Vectors of Large Matrices , 1983 .

[12]  Efstratios Gallopoulos,et al.  Computing smallest singular triplets with implicitly restarted Lanczos bidiagonalization , 2004, Applied Numerical Mathematics.

[13]  Gilbert Strang,et al.  The Potential Theory of Several Intervals and Its Applications , 2001 .

[14]  Zhongxiao Jia,et al.  A Refined Harmonic Lanczos Bidiagonalization Method and an Implicitly Restarted Algorithm for Computing the Smallest Singular Triplets of Large Matrices , 2009, SIAM J. Sci. Comput..

[15]  Zhongxiao Jia,et al.  An Implicitly Restarted Refined Bidiagonalization Lanczos Method for Computing a Partial Singular Value Decomposition , 2003, SIAM J. Matrix Anal. Appl..

[16]  Chao Yang,et al.  ARPACK users' guide - solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods , 1998, Software, environments, tools.

[17]  Michiel E. Hochstenbach,et al.  Harmonic and Refined Extraction Methods for the Singular Value Problem, with Applications in Least Squares Problems , 2004 .

[18]  Kesheng Wu,et al.  Dynamic Thick Restarting of the Davidson, and the Implicitly Restarted Arnoldi Methods , 1998, SIAM J. Sci. Comput..

[19]  JESSE L. BARLOW,et al.  Reorthogonalization for the Golub–Kahan–Lanczos bidiagonal reduction , 2013, Numerische Mathematik.

[20]  Datian Niu,et al.  A harmonic Lanczos bidiagonalization method for computing interior singular triplets of large matrices , 2012, Appl. Math. Comput..

[21]  Gene H. Golub,et al.  An Inverse Free Preconditioned Krylov Subspace Method for Symmetric Generalized Eigenvalue Problems , 2002, SIAM J. Sci. Comput..

[22]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[23]  Lothar Reichel,et al.  Restarted block Lanczos bidiagonalization methods , 2007, Numerical Algorithms.

[24]  L. Trefethen,et al.  Numerical linear algebra , 1997 .

[25]  Lothar Reichel,et al.  Augmented Implicitly Restarted Lanczos Bidiagonalization Methods , 2005, SIAM J. Sci. Comput..

[26]  Michiel E. Hochstenbach,et al.  A Jacobi-Davidson Type SVD Method , 2001, SIAM J. Sci. Comput..

[27]  R. Morgan Computing Interior Eigenvalues of Large Matrices , 1991 .

[28]  B. Parlett The Symmetric Eigenvalue Problem , 1981 .

[29]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[30]  Andreas Stathopoulos,et al.  Nearly Optimal Preconditioned Methods for Hermitian Eigenproblems under Limited Memory. Part I: Seeking One Eigenvalue , 2007, SIAM J. Sci. Comput..

[31]  Michael W. Berry,et al.  Large-Scale Sparse Singular Value Computations , 1992 .

[32]  Lothar Reichel,et al.  An implicitly restarted block Lanczos bidiagonalization method using Leja shifts , 2012 .

[33]  Vicente Hernández,et al.  A robust and efficient parallel SVD solver based on restarted Lanczos bidiagonalization. , 2007 .

[34]  R. Larsen Lanczos Bidiagonalization With Partial Reorthogonalization , 1998 .

[35]  Henk A. van der Vorst,et al.  Approximate solutions and eigenvalue bounds from Krylov subspaces , 1995, Numer. Linear Algebra Appl..

[36]  Z. Jia,et al.  Refined iterative algorithms based on Arnoldi's process for large unsymmetric eigenproblems , 1997 .

[37]  Qiao Liang,et al.  Computing Singular Values of Large Matrices with an Inverse-Free Preconditioned Krylov Subspace Method , 2014 .

[38]  Franklin T. Luk,et al.  A Block Lanczos Method for Computing the Singular Values and Corresponding Singular Vectors of a Matrix , 1981, TOMS.

[39]  R. Morgan,et al.  Harmonic projection methods for large non-symmetric eigenvalue problems , 1998 .

[40]  Gene H. Golub,et al.  Calculating the singular values and pseudo-inverse of a matrix , 2007, Milestones in Matrix Computation.