PRIMME_SVDS: A High-Performance Preconditioned SVD Solver for Accurate Large-Scale Computations

The increasing number of applications requiring the solution of large scale singular value problems have rekindled interest in iterative methods for the SVD. Some promising recent ad- vances in large scale iterative methods are still plagued by slow convergence and accuracy limitations for computing smallest singular triplets. Furthermore, their current implementations in MATLAB cannot address the required large problems. Recently, we presented a preconditioned, two-stage method to effectively and accurately compute a small number of extreme singular triplets. In this research, we present a high-performance software, PRIMME SVDS, that implements our hybrid method based on the state-of-the-art eigensolver package PRIMME for both largest and smallest singular values. PRIMME SVDS fills a gap in production level software for computing the partial SVD, especially with preconditioning. The numerical experiments demonstrate its superior performance compared to other state-of-the-art software and its good parallel performance under strong and weak scaling.

[1]  G. Stewart.  Matrix Algorithms, Volume II: Eigensystems , 2001 .

[2]  Zhongxiao Jia,et al.  A Refined Harmonic Lanczos Bidiagonalization Method and an Implicitly Restarted Algorithm for Computing the Smallest Singular Triplets of Large Matrices , 2009, SIAM J. Sci. Comput..

[3]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[4]  Z. Jia,et al.  Refined iterative algorithms based on Arnoldi's process for large unsymmetric eigenproblems , 1997 .

[5]  Qiao Liang,et al.  Computing Singular Values of Large Matrices with an Inverse-Free Preconditioned Krylov Subspace Method , 2014 .

[6]  Zhongxiao Jia,et al.  An Implicitly Restarted Refined Bidiagonalization Lanczos Method for Computing a Partial Singular Value Decomposition , 2003, SIAM J. Matrix Anal. Appl..

[7]  Min Zeng,et al.  Harmonic projection methods for large non-symmetric eigenvalue problems , 1998, Numer. Linear Algebra Appl..

[8]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[9]  Lothar Reichel,et al.  Augmented Implicitly Restarted Lanczos Bidiagonalization Methods , 2005, SIAM J. Sci. Comput..

[10]  Andreas Stathopoulos,et al.  A Case for a Biorthogonal Jacobi-Davidson Method: Restarting and Correction Equation , 2002, SIAM J. Matrix Anal. Appl..

[11]  Michael W. Berry,et al.  Large-Scale Sparse Singular Value Computations , 1992 .

[12]  J. Cullum,et al.  A Lanczos Algorithm for Computing Singular Values and Vectors of Large Matrices , 1983 .

[13]  Efstratios Gallopoulos,et al.  Computing smallest singular triplets with implicitly restarted Lanczos bidiagonalization , 2004, Applied Numerical Mathematics.

[14]  Michael A. Heroux,et al.  Tpetra, and the use of generic programming in scientific computing , 2012, Sci. Program..

[15]  Andreas Stathopoulos,et al.  Nearly Optimal Preconditioned Methods for Hermitian Eigenproblems under Limited Memory. Part I: Seeking One Eigenvalue , 2007, SIAM J. Sci. Comput..

[16]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[17]  Miloud Sadkane,et al.  Computation of the fundamental singular subspace of a large matrix , 1997 .

[18]  Andreas Stathopoulos,et al.  A Preconditioned Hybrid SVD Method for Accurately Computing Singular Triplets of Large Matrices , 2015, SIAM J. Sci. Comput..

[19]  Michiel E. Hochstenbach,et al.  A Jacobi-Davidson Type SVD Method , 2001, SIAM J. Sci. Comput..

[20]  R. Morgan.  Computing Interior Eigenvalues of Large Matrices , 1991 .

[21]  Andreas Stathopoulos,et al.  PRIMME: preconditioned iterative multimethod eigensolver—methods and software description , 2010, TOMS.

[22]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[23]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[24]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[25]  Vicente Hernández,et al.  SLEPc: A scalable and flexible toolkit for the solution of eigenvalue problems , 2005, TOMS.

[26]  Matthias W. Seeger,et al.  Using the Nyström Method to Speed Up Kernel Machines , 2000, NIPS.

[27]  Gene H. Golub,et al.  Calculating the singular values and pseudo-inverse of a matrix , 2007, Milestones in Matrix Computation.

[28]  Michele Martone,et al.  Efficient multithreaded untransposed, transposed or symmetric sparse matrix-vector multiplication with the Recursive Sparse Blocks format , 2014, Parallel Comput..

[29]  Lloyd N. Trefethen,et al.  Computation of pseudospectra , 1999, Acta Numerica.

[30]  J. Dongarra.  Improving the Accuracy of Computed Singular Values , 1983 .

[31]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[32]  R. Larsen.  Lanczos Bidiagonalization With Partial Reorthogonalization , 1998 .

[33]  Chao Yang,et al.  ARPACK users' guide - solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods , 1998, Software, environments, tools.

[34]  Michiel E. Hochstenbach,et al.  Harmonic and Refined Extraction Methods for the Singular Value Problem, with Applications in Least Squares Problems , 2004 .

[35]  Gene H. Golub,et al.  An Inverse Free Preconditioned Krylov Subspace Method for Symmetric Generalized Eigenvalue Problems , 2002, SIAM J. Sci. Comput..

[36]  Sivan Toledo,et al.  Spectral condition‐number estimation of large sparse matrices , 2013, Numer. Linear Algebra Appl..

[37]  William W. Cohen,et al.  Power Iteration Clustering , 2010, ICML.

[38]  Lothar Reichel,et al.  Restarted block Lanczos bidiagonalization methods , 2007, Numerical Algorithms.

[39]  V. E. Henson,et al.  BoomerAMG: a parallel algebraic multigrid solver and preconditioner , 2002 .

[40]  Brian Kingsbury,et al.  Efficient one-vs-one kernel ridge regression for speech recognition , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[41]  Jie Chen,et al.  Revisiting Random Binning Features: Fast Convergence and Strong Parallelizability , 2016, KDD.

[42]  B. AfeArd.  CALCULATING THE SINGULAR VALUES AND PSEUDOINVERSE OF A MATRIX , 2022 .

[43]  G. W. Stewart,et al.  Matrix algorithms , 1998 .

[44]  Lingfei Wu,et al.  Estimating the trace of the matrix inverse by interpolating from the diagonal of an approximate inverse , 2015, J. Comput. Phys..

[45]  Lothar Reichel,et al.  An implicitly restarted block Lanczos bidiagonalization method using Leja shifts , 2012 .

[46]  Vicente Hernández,et al.  A robust and efficient parallel SVD solver based on restarted Lanczos bidiagonalization. , 2007 .

[47]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.