Stable and Efficient Spectral Divide and Conquer Algorithms for the Symmetric Eigenvalue Decomposition and the SVD

Spectral divide and conquer algorithms solve the eigenvalue problem for all the eigenvalues and eigenvectors by recursively computing an invariant subspace for a subset of the spectrum and using it to decouple the problem into two smaller subproblems. A number of such algorithms have been developed over the last 40 years, often motivated by parallel computing and, most recently, with the aim of achieving minimal communication costs. However, none of the existing algorithms has been proved to be backward stable, and they all have a significantly higher arithmetic cost than the standard algorithms currently used. We present new spectral divide and conquer algorithms for the symmetric eigenvalue problem and the singular value decomposition that are backward stable, achieve lower bounds on communication costs recently derived by Ballard, Demmel, Holtz, and Schwartz, and have operation counts within a small constant factor of those for the standard algorithms. The new algorithms are built on the polar decompos...

[1]  V. Strassen.  Gaussian elimination is not optimal , 1969 .

[2]  E. Denman,et al.  A computational method for eigenvalues and eigenvectors of a matrix with real eigenvalues , 1973 .

[3]  E. Denman,et al.  The matrix sign function and computations in systems , 1976 .

[4]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[5]  Tony F. Chan,et al.  An Improved Algorithm for Computing the Singular Value Decomposition , 1982, TOMS.

[6]  Gene H. Golub,et al.  Matrix computations , 1983 .

[7]  N. Higham.  Computing the polar decomposition with applications , 1986 .

[8]  S. Godunov.  Problem of the dichotomy of the spectrum of a matrix , 1986 .

[9]  S. Godunov,et al.  Circular dichotomy of the spectrum of a matrix , 1988 .

[10]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[11]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[12]  James Demmel,et al.  Stability of block algorithms with fast level-3 BLAS , 1992, TOMS.

[13]  L. Auslander,et al.  On parallelizable eigensolvers , 1992 .

[14]  A. Malyshev.  Parallel Algorithm for Solving Some Spectral Problems of Linear Algebra , 1993 .

[15]  James Demmel,et al.  Design of a Parallel Nonsymmetric Eigenroutine Toolbox, Part I , 1993, PPSC.

[16]  Xiaobai Sun,et al.  The PRISM project: infrastructure and algorithms for parallel eigensolvers , 1993, Proceedings of Scalable Parallel Libraries Conference.

[17]  N. Higham.  The matrix sign decomposition and its relation to the polar decomposition , 1994 .

[18]  Nicholas J. Higham,et al.  A NEW PARALLEL ALGORITHM FOR COMPUTING THE SINGULAR-VALUE DECOMPOSITION , 1994 .

[19]  J. Demmel,et al.  Inverse Free Parallel Spectral Divide and Conquer Algorithms for Nonsymmetric Eigenproblems , 1994 .

[20]  Stanley C. Eisenstat,et al.  A Divide-and-Conquer Algorithm for the Symmetric Tridiagonal Eigenproblem , 1995, SIAM J. Matrix Anal. Appl..

[21]  Gene H. Golub,et al.  Matrix Computations, Third Edition , 1996 .

[22]  James Demmel,et al.  The Spectral Decomposition of Nonsymmetric Matrices on Distributed Memory Parallel Computers , 1997, SIAM J. Sci. Comput..

[23]  Steven Huss-Lederman,et al.  A Parallelizable Eigensolver for Real Diagonalizable Matrices with Real Eigenvalues , 1997, SIAM J. Sci. Comput..

[24]  J. Demmel,et al.  Using the Matrix Sign Function to Compute Invariant Subspaces , 1998, SIAM J. Matrix Anal. Appl..

[25]  S. Godunov.  Modern Aspects of Linear Algebra , 1998 .

[26]  H. Zha,et al.  A Cubically Convergent Parallelizable Method for the Hermitian Eigenvalue Problem , 1998 .

[27]  James Demmel,et al.  LAPACK Users' Guide, Third Edition , 1999, Software, Environments and Tools.

[28]  Jack Dongarra,et al.  LAPACK Users' Guide, 3rd ed. , 1999 .

[29]  Jack Dongarra,et al.  Templates for the Solution of Algebraic Eigenvalue Problems , 2000, Software, environments, tools.

[30]  Enrique S. Quintana-Ortí,et al.  Parallel Spectral Division Using the Matrix Sign Function for the Generalized Eigenproblem , 2000, Int. J. High Speed Comput..

[31]  Nicholas J. Higham,et al.  A Block Algorithm for Matrix 1-Norm Estimation, with an Application to 1-Norm Pseudospectra , 2000, SIAM J. Matrix Anal. Appl..

[32]  Inderjit S. Dhillon,et al.  Orthogonal Eigenvectors and Relative Gaps , 2003, SIAM J. Matrix Anal. Appl..

[33]  Nicholas J. Higham,et al.  Parallel Singular Value Decomposition via the Polar Decomposition , 2006 .

[34]  Enrique S. Quintana-Ortí,et al.  Specialized Spectral Division Algorithms for Generalized Eigenproblems Via the Inverse-Free Iteration , 2006, PARA.

[35]  Ying,et al.  FAST PARALLELIZABLE METHODS FOR COMPUTING INVARIANT SUBSPACES OF HERMITIAN MATRICES , 2007 .

[36]  James Demmel,et al.  Fast linear algebra is stable , 2006, Numerische Mathematik.

[37]  James Demmel,et al.  Algorithm 880: A testing infrastructure for symmetric tridiagonal eigensolvers , 2008, TOMS.

[38]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[39]  Ralph Byers,et al.  A New Scaling for Newton's Iteration for the Polar Decomposition and its Backward Stability , 2008, SIAM J. Matrix Anal. Appl..

[40]  James Demmel,et al.  Minimizing Communication for Eigenproblems and the Singular Value Decomposition , 2010, ArXiv.

[41]  Paul Willems,et al.  On MR3-type Algorithms for the Tridiagonal Symmetric Eigenproblem and the Bidiagonal SVD , 2018 .

[42]  F. Mathematik.  On MR 3 -type Algorithms for the Tridiagonal Symmetric Eigenproblem and the Bidiagonal SVD , 2010 .

[43]  Zhaojun Bai,et al.  Optimizing Halley's Iteration for Computing the Matrix Polar Decomposition , 2010, SIAM J. Matrix Anal. Appl..

[44]  James Demmel,et al.  Communication-optimal Parallel and Sequential Cholesky Decomposition , 2009, SIAM J. Sci. Comput..

[45]  James Demmel,et al.  CALU: A Communication Optimal LU Factorization Algorithm , 2011, SIAM J. Matrix Anal. Appl..

[46]  James Demmel,et al.  Minimizing Communication in Numerical Linear Algebra , 2009, SIAM J. Matrix Anal. Appl..

[47]  James Demmel,et al.  Communication-optimal Parallel and Sequential QR and LU Factorizations , 2008, SIAM J. Sci. Comput..

[48]  Nicholas J. Higham,et al.  Backward Stability of Iterations for Computing the Polar Decomposition , 2012, SIAM J. Matrix Anal. Appl..

[49]  James Demmel,et al.  LU Factorization with Panel Rank Revealing Pivoting and Its Communication Avoiding Version , 2012, SIAM J. Matrix Anal. Appl..