Blocking and parallelization of the Hari-Zimmermann variant of the Falk-Langemeyer algorithm for the generalized SVD

New parallel Jacobi-type algorithm for the generalized singular value problem.The algorithm is 15 times faster than DGGSVD from LAPACK.It is also more accurate. The paper describes how to modify the two-sided Hari-Zimmermann algorithm for computation of the generalized eigenvalues of a matrix pair (A, B), where B is positive definite, to an implicit algorithm that computes the generalized singular values of a pair (F, G). In addition, we present blocking and parallelization techniques for speedup of the computation.For triangular matrix pairs of a moderate size, numerical tests show that the double precision sequential pointwise algorithm is several times faster than the Lapack DTGSJA algorithm, while the accuracy is slightly better, especially for small generalized singular values.Cache-aware algorithms, implemented either as the block-oriented, or as the full block algorithm, are several times faster than the pointwise algorithm. The algorithm is almost perfectly parallelizable, so parallel shared memory versions of the algorithm are perfectly scalable, and their speedup almost solely depends on the number of cores used. A hybrid shared/distributed memory algorithm is intended for huge matrices that do not fit into the shared memory.

[1]  James Demmel,et al.  Computing the Generalized Singular Value Decomposition , 1993, SIAM J. Sci. Comput..

[2]  Vedran Novakovic,et al.  A Hierarchically Blocked Jacobi SVD Algorithm for Single and Multiple Graphics Processing Units , 2014, SIAM Journal on Scientific Computing.

[3]  Pradip Kumar Bhuyan,et al.  Application of generalized singular value decomposition to ionospheric tomography , 2004 .

[4]  Vjeran Hari,et al.  Block-oriented J-Jacobi methods for Hermitian matrices , 2010 .

[5]  C. Loan.  Computing the CS and the generalized singular value decompositions , 1985 .

[6]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[7]  Vedran Novakovic,et al.  Three-level parallel J-Jacobi algorithms for Hermitian matrices , 2010, Appl. Math. Comput..

[8]  Vjeran Hari,et al.  Convergence to diagonal form of block Jacobi-type methods , 2015, Numerische Mathematik.

[9]  J. Bunch,et al.  Direct Methods for Solving Symmetric Indefinite Systems of Linear Equations , 1971 .

[10]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[11]  Josip Matejas.  Accuracy of one step of the Falk-Langemeyer method , 2014, Numerical Algorithms.

[12]  G. Stewart.  Computing theCS decomposition of a partitioned orthonormal matrix , 1982 .

[13]  Vjeran Hari,et al.  On the quadratic convergence of the Falk-Langemeyer method , 1991 .

[14]  P. P. Rijk.  A one-sided Jacobi algorithm for computing the singular value decomposition on avector computer , 1989 .

[15]  Weichung Yeih,et al.  APPLICATIONS OF THE GENERALIZED SINGULAR-VALUE DECOMPOSITION METHOD ON THE EIGENPROBLEM USING THE INCOMPLETE BOUNDARY ELEMENT FORMULATION , 2000 .

[16]  Franklin T. Luk,et al.  A Proof of Convergence for Two Parallel Jacobi SVD Algorithms , 1989, IEEE Trans. Computers.

[17]  Vedran Novakovic,et al.  A GPU-based hyperbolic SVD algorithm , 2010, ArXiv.

[18]  D. Botstein,et al.  Generalized singular value decomposition for comparative analysis of genome-scale expression data sets of two different organisms , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[19]  K. Zimmermann,et al.  Zur Konvergenz eines Jacobiverfahrens für gewöhnliche und verallgemeinerte Eigenwertprobleme , 1969 .

[20]  M. Saunders,et al.  Towards a Generalized Singular Value Decomposition , 1981 .

[21]  C. Loan.  Generalizing the Singular Value Decomposition , 1976 .

[22]  Z. Drmač.  A Tangent Algorithm for Computing the Generalized Singular Value Decomposition , 1998 .

[23]  Z. Drmač.  A posteriori computation of the singular vectors in a preconditioned Jacobi SVD algorithm , 1999 .

[24]  Ivan Slapničar,et al.  Componentwise Analysis of Direct Factorization of Real Symmetric and Hermitian Matrices , 1998 .

[25]  Vedran Novakovic,et al.  Novel modifications of parallel Jacobi algorithms , 2011, Numerical Algorithms.

[26]  Sanja Singer,et al.  Full block J-Jacobi method for Hermitian matrices , 2014 .

[27]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[28]  Zhaojun Bai,et al.  A New Preprocessing Algorithm for the Computation of the Generalized Singular Value Decomposition , 1993, SIAM J. Sci. Comput..

[29]  Brian D. Sutton.  Stable Computation of the CS Decomposition: Simultaneous Bidiagonalization , 2012, SIAM J. Matrix Anal. Appl..

[30]  Günther Gose.  Das Jacobi-Verfahren für Ax = λBx , 1979 .