Design of a Parallel Nonsymmetric Eigenroutine Toolbox, Part I

The dense nonsymmetric eigenproblem is one of the hardest linear algebra problems to solve effectively on massively parallel machines. Rather than trying to design a ``black box'''' eigenroutine in the spirit of EISPACK or LAPACK, we propose building a toolbox for this problem. The tools are meant to be used in different combinations on different problems and architectures. In this paper, we will describe these tools which include basic block matrix computations, the matrix sign function, 2-dimensional bisection, and spectral divide and conquer using the matrix sign function to find selected eigenvalues. We also outline how we deal with ill-conditioning and potential instability. Numerical examples are included. A future paper will discuss error analysis in detail and extensions to the generalized eigenproblem.

[1]  Al Geist.  Parallel Tridiagonalization of a General Matrix Using Distributed-Memory Multiprocessors , 1989, PPSC.

[2]  Peter Lancaster,et al.  The theory of matrices , 1969 .

[3]  P. J. Eberlein.  Errata: A Jacobi-Like Method for the Automatic Computation of Eigenvalues and Eigenvectors , 1962 .

[4]  Charles Kenney,et al.  Polar Decomposition and Matrix Sign Function Condition Estimates , 1991, SIAM J. Sci. Comput..

[5]  J. Demmel.  On condition numbers and the distance to the nearest ill-posed problem , 2015 .

[6]  R. van de Geijn,et al.  A look at scalable dense linear algebra libraries , 1992, Proceedings Scalable High Performance Computing Conference SHPCC-92..

[7]  P. Henrici,et al.  Circular arithmetic and the determination of polynomial zeros , 1971 .

[8]  Gautam M. Shroff.  A parallel algorithm for the eigenvalues and eigenvectors of a general complex matrix , 1990 .

[9]  J. Dongarra,et al.  A Parallel Algorithm for the Non-Symmetric Eigenvalue Problem , 1991 .

[10]  J. L. Howland.  The sign matrix and the separation of matrix eigenvalues , 1983 .

[11]  T. Chan.  Rank revealing QR factorizations , 1987 .

[12]  Danny C. Sorensen,et al.  Implicit Application of Polynomial Filters in a k-Step Arnoldi Method , 1992, SIAM J. Matrix Anal. Appl..

[13]  R. Freund,et al.  QMR: a quasi-minimal residual method for non-Hermitian linear systems , 1991 .

[14]  K. Veselic,et al.  A quadratically convergent Jacobi-like method for real matrices with complex eigenvalues , 1979 .

[15]  R. Byers.  A Bisection Method for Measuring the Distance of a Stable Matrix to the Unstable Matrices , 1988 .

[16]  F. Chatelin.  Simultaneous Newton’s Iteration for the Eigenproblem , 1984 .

[17]  Jack J. Dongarra,et al.  Matrix Eigensystem Routines — EISPACK Guide Extension , 1977, Lecture Notes in Computer Science.

[18]  J. H. Wilkinson,et al.  IMPROVING THE ACCURACY OF COMPUTED EIGENVALUES AND EIGENVECTORS , 1983 .

[19]  Nancy Nichols,et al.  On the stability radius of a generalized state-space system , 1993 .

[20]  Patricia J. Eberlein,et al.  On the Schur Decomposition of a Matrix for Parallel Computation , 1985, IEEE Transactions on Computers.

[21]  V. N. Bogaevski,et al.  Matrix Perturbation Theory , 1991 .

[22]  R. Byers.  Numerical Stability and Instability in Matrix Sign Function Based Algorithms , 1986 .

[23]  P. Dooren.  The Computation of Kronecker's Canonical Form of a Singular Pencil , 1979 .

[24]  James Demmel,et al.  On a Block Implementation of Hessenberg Multishift QR Iteration , 1989, Int. J. High Speed Comput..

[25]  J. Demmel,et al.  On swapping diagonal blocks in real Schur form , 1993 .

[26]  L. Balzer.  Accelerated convergence of the matrix sign function method of solving Lyapunov, Riccati and other matrix equations , 1980 .

[27]  C. Bischof,et al.  Robust incremental condition estimation , 1991 .

[28]  Tosio Kato.  Perturbation theory for linear operators , 1966 .

[29]  P. Dooren,et al.  An improved algorithm for the computation of Kronecker's canonical form of a singular pencil , 1988 .

[30]  Jack J. Dongarra,et al.  A set of level 3 basic linear algebra subprograms , 1990, TOMS.

[31]  S. Lennart Johnsson,et al.  Block-Cyclic Dense Linear Algebra , 1993, SIAM J. Sci. Comput..

[32]  G. Stewart.  A Jacobi-Like Algorithm for Computing the Schur Decomposition of a Nonhermitian Matrix , 1985 .

[33]  Nicholas J. Highham.  A survey of condition number estimation for triangular matrices , 1987 .

[34]  J. Demmel.  Trading Off Parallelism and Numerical Stability , 1992 .

[35]  A. Laub,et al.  Rational iterative methods for the matrix sign function , 1991 .

[37]  R. Byers.  Solving the algebraic Riccati equation with the matrix sign function , 1987 .

[38]  G. Stewart.  Error and Perturbation Bounds for Subspaces Associated with Certain Eigenvalue Problems , 1973 .

[39]  A. J. Geurts,et al.  A contribution to the theory of condition , 1982 .

[40]  Robert A. van de Geijn.  Implementing the qr-algorithm on an array of processors , 1987 .

[41]  A. Malyshev.  Parallel Algorithm for Solving Some Spectral Problems of Linear Algebra , 1993 .

[42]  T. Y. Li,et al.  Solving eigenvalue problems of real nonsymmetric matrices with real homotopies , 1992 .

[43]  S. Batterson.  Convergence of the shifted QR algorithm on 3×3 normal matrices , 1990 .

[44]  M. Paardekooper,et al.  A quadratically convergent parallel Jacobi process for diagonally dominant matrices with distinct eigenvalues , 1990 .

[45]  F. Chatelin.  Valeurs propres de matrices , 1988 .

[46]  J. Demmel,et al.  Stably Computing the Kronecker Structure and Reducing Subspaces of Singular Pencils A-λ for Uncertain Data , 1986 .

[47]  Ed Anderson,et al.  LAPACK users' guide - [release 1.0] , 1992 .

[48]  L. Auslander,et al.  On parallelizable eigensolvers , 1992 .

[49]  J. Leyva-Ramos,et al.  Spectral decomposition of a matrix using the generalized sign matrix , 1981 .

[50]  David S. Watkins,et al.  Convergence of algorithms of decomposition type for the eigenvalue problem , 1991 .

[51]  C. Bischof,et al.  A divide-and-conquer method for tridiagonalizing symmetric matrices with repeated eigenvalues , 1994 .

[52]  Judith Gardiner,et al.  A generalization of the matrix sign function solution for algebraic Riccati equations , 1985, 1985 24th IEEE Conference on Decision and Control.

[53]  Ed Anderson,et al.  LAPACK Users' Guide , 1995 .

[54]  David S. Watkins,et al.  Shifting Strategies for the Parallel QR Algorithm , 1994, SIAM J. Sci. Comput..

[55]  S. Godunov,et al.  Circular dichotomy of the spectrum of a matrix , 1988 .

[56]  G. W. Stewart,et al.  A parallel implementation of the QR-algorithm , 1987, Parallel Comput..

[57]  N. Higham.  The matrix sign decomposition and its relation to the polar decomposition , 1994 .

[58]  M. Chu.  A note on the homotopy method for linear algebraic eigenvalue problems , 1988 .

[59]  Al Geist,et al.  Finding eigenvalues and eigenvectors of unsymmetric matrices using a distributed-memory multiprocessor , 1990, Parallel Comput..

[60]  E. Stickel.  Separating eigenvalues using the matrix sign function , 1991 .

[61]  N. Higham.  Computing the polar decomposition with applications , 1986 .

[62]  A. Laub.  Invariant Subspace Methods for the Numerical Solution of Riccati Equations , 1991 .

[63]  A. Edelman.  On the distribution of a scaled condition number , 1992 .

[64]  Alan J. Laub,et al.  On Scaling Newton's Method for Polar Decomposition and the Matrix Sign Function , 1990, 1990 American Control Conference.

[65]  S. Boyd,et al.  A regularity result for the singular values of a transfer matrix and a quadratically convergent algorithm for computing its L ∞ -norm , 1990 .

[66]  Gene H. Golub,et al.  Matrix computations , 1983 .

[67]  E. Denman,et al.  The matrix sign function and computations in systems , 1976 .

[68]  Stephen P. Boyd,et al.  A bisection method for computing the H∞ norm of a transfer matrix and related problems , 1989, Math. Control. Signals Syst..

[69]  J. D. Roberts,et al.  Linear model reduction and solution of the algebraic Riccati equation by use of the sign function , 1980 .

[70]  Robert A. van de Geijn.  A Novel Storage Scheme for Parallel Jacobi Methods , 1988 .

[71]  J. Cullum,et al.  A Practical Procedure for Computing Eigenvalues of Large Sparse Nonsymmetric Matrices , 1986 .

[72]  Y. Saad,et al.  Numerical solution of large nonsymmetric eigenvalue problems , 1989 .

[73]  A. Sameh.  On Jacobi and Jacobi-like algorithms for a parallel computer , 1971 .

[74]  Tien-Yien Li,et al.  Homotopy-determinant algorithm for solving nonsymmetric eigenvalue problems , 1992 .

[75]  Bo Kågström,et al.  An Algorithm for Numerical Computation of the Jordan Normal Form of a Complex Matrix , 1980, TOMS.

[76]  N. Higham,et al.  Stability of methods for matrix inversion , 1992 .

[77]  C. Hermite,et al.  On the number of roots of an algebraic equation contained between given limits , 1977 .

[78]  E. Denman,et al.  A computational method for eigenvalues and eigenvectors of a matrix with real eigenvalues , 1973 .

[79]  Jack Dongarra,et al.  Computing the eigenvalues and eigenvectors of a general matrix by reduction to general tridiagonal form , 1990 .

[80]  George A. Geist.  Reduction of a general matrix to tridiagonal form using a hypercube multiprocessor , 1991 .

[81]  Tien-Yien Li,et al.  Homotopy method for general l-matrix problems , 1988 .

[82]  A. N. Malychev.  Parallel aspects of some spectral problems in linear algebra , 1991 .

[83]  James Demmel,et al.  Parallel numerical linear algebra , 1993, Acta Numerica.