Large-Scale Sparse Singular Value Computations

We present four numerical methods for computing the singular value decomposition (SVD) of large sparse matrices on a multiprocessor architecture. We emphasize Lanczos and subspace iteration-based methods for determining several of the largest singular triplets (singular values and corresponding left- and right-singular vectors) for sparse matrices arising from two practical applications: information retrieval and seismic reflection tomography. The target architectures for our implementations are the CRAY-2S/4–128 and Alliant FX/80. The sparse SVD problem is well motivated by recent information-retrieval techniques in which dominant singular values and their corresponding singular vectors of large sparse term-document matrices are desired, and by nonlinear inverse problems from seismic tomography applications which require approximate pseudo-inverses of large sparse Jacobian matrices. This research may help advance the development of future out-of-core sparse SVD methods, which can be used, for example, to handle extremely large sparse matrices 0 × (106) rows or columns associated with extremely large databases in query-based information-retrieval applications.

[1]  F. L. Bauer.  Das Verfahren der Treppeniteration und verwandte Verfahren zur Lösung algebraischer Eigenwertprobleme , 1957 .

[2]  M. Hestenes.  Inversion of Matrices by Biorthogonalization and Related Results , 1958 .

[3]  L. Mirsky.  SYMMETRIC GAUGE FUNCTIONS AND UNITARILY INVARIANT NORMS , 1960 .

[4]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[5]  J. Gillis,et al.  Linear Differential Operators , 1963 .

[6]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[7]  S. Kaniel.  Estimates for Some Computational Techniques - in Linear Algebra , 1966 .

[8]  J. H. Wilkinson,et al.  The implicitQL algorithm , 1968 .

[9]  H. Rutishauser.  Computational aspects of F. L. Bauer's simultaneous iteration method , 1969 .

[10]  H. Rutishauser.  Simultaneous iteration method for symmetric matrices , 1970 .

[11]  J. H. Wilkinson,et al.  The implicit QL algorithm , 1971 .

[12]  E. Polak.  Introduction to linear and nonlinear programming , 1973 .

[13]  Richard R. Underwood.  An iterative block Lanczos method for the solution of large sparse symmetric eigenproblems , 1975 .

[14]  C. Paige.  Error Analysis of the Lanczos Algorithm for Tridiagonalizing a Symmetric Matrix , 1976 .

[15]  Brian T. Smith,et al.  Matrix Eigensystem Routines — EISPACK Guide , 1974, Lecture Notes in Computer Science.

[16]  B. Parlett,et al.  The Lanczos algorithm with selective orthogonalization , 1979 .

[17]  J. H. Wilkinson,et al.  Inverse Iteration, Ill-Conditioned Equations and Newton’s Method , 1979 .

[18]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[19]  Y. Saad.  On the Rates of Convergence of the Lanczos and the Block-Lanczos Methods , 1980 .

[20]  John Aruthur Wisniewski.  On Solving the Large Sparse Generalized Eigenvalue Problem , 1981 .

[21]  Franklin T. Luk,et al.  A Block Lanczos Method for Computing the Singular Values and Corresponding Singular Vectors of a Matrix , 1981, TOMS.

[22]  John G. Lewis,et al.  Sparse matrix test problems , 1982, SGNM.

[23]  Gene H. Golub,et al.  Matrix computations , 1983 .

[24]  H. Simon.  Analysis of the symmetric Lanczos algorithm with reorthogonalization methods , 1984 .

[25]  Jack J. Dongarra,et al.  A fast algorithm for the symmetric eigenvalue problem , 1985, 1985 IEEE 7th Symposium on Computer Arithmetic (ARITH).

[26]  J. Cullum,et al.  Lanczos algorithms for large symmetric eigenvalue computations , 1985 .

[27]  Jack J. Dongarra,et al.  A fully parallel algorithm for the symmetric eigenvalue problem , 1985, PPSC.

[28]  Ahmed H. Sameh,et al.  A multiprocessor algorithm for the symmetric tridiagonal eigenvalue problem , 1985, PPSC.

[29]  Michael W. Berry,et al.  Multiprocessor Jacobi Algorithms for Dense Symmetric Eigenvalue and Singular Value Decompositions , 1986, ICPP.

[30]  William Jalby,et al.  The use of BLAS3 in linear algebra on a parallel processor with a hierarchical memory , 1987 .

[31]  Jack Dongarra,et al.  LINPACK Users' Guide , 1987 .

[32]  R. P. Bording,et al.  Applications of seismic travel-time tomography , 1987 .

[33]  Ahmed H. Sameh,et al.  Trace Minimization Algorithm for the Generalized Eigenvalue Problem , 1982, PPSC.

[34]  S. T. Dumais,et al.  Using latent semantic analysis to improve access to textual information , 1988, CHI '88.

[35]  J. Scales,et al.  Robust methods in inverse theory , 1988 .

[36]  Jack J. Dongarra,et al.  An extended set of FORTRAN basic linear algebra subprograms , 1988, TOMS.

[37]  A. Sameh,et al.  An overview of parallel algorithms for the singular value and symmetric eigenvalue problems , 1989 .

[38]  Y. Saad,et al.  Krylov Subspace Methods on Supercomputers , 1989 .

[39]  J. Scales,et al.  Regularisation of nonlinear inverse problems: imaging the near-surface weathering layer , 1990 .

[40]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[41]  Michael W. Berry.  Multiprocessor sparse SVD algorithms and applications , 1991 .