Recent Developments in Dense Numerical Linear Algebra Recent Developments in Dense Numerical Linear Algebra

We survey recent developments in dense numerical linear algebra, covering linear systems, least squares problems and eigenproblems. Topics considered include the design and analysis of block, partitioned and parallel algorithms, condition number estimation, componentwise error analysis , and the computation of practical error bounds. Frequent reference is made to LAPACK, the state of the art package of Fortran software designed to solve linear algebra problems eeciently and accurately on high-performance computers.

[1]  G. Schulz.  Iterative Berechung der reziproken Matrix , 1933 .

[2]  An Automatic Computing Engine for the National Physical Laboratory , 1946, Nature.

[3]  James Hardy Wilkinson.  The automatic computing engine at the National Physical Laboratory , 1948, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[4]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[5]  W. Kahan.  Accurate eigenvalues of a symmetric tri-diagonal matrix , 1966 .

[6]  C. Lawson,et al.  Extensions and applications of the Householder algorithm for solving linear least squares problems , 1969 .

[7]  G. Stewart,et al.  On the Numerical Properties of an Iterative Method for Computing the Moore–Penrose Generalized Inverse , 1974 .

[8]  J. Bunch,et al.  Some stable methods for calculating inertia and solving symmetric linear systems , 1977 .

[9]  L. Csanky,et al.  Fast parallel matrix inversion algorithms , 1975, 16th Annual Symposium on Foundations of Computer Science (sfcs 1975).

[10]  G. Stewart,et al.  A Stable Variant of the Secant Method for Solving Nonlinear Equations , 1976 .

[11]  B. S. Garbow,et al.  Matrix Eigensystem Routines — EISPACK Guide , 1974, Lecture Notes in Computer Science.

[12]  Jack J. Dongarra,et al.  Matrix Eigensystem Routines — EISPACK Guide Extension , 1977, Lecture Notes in Computer Science.

[13]  H. Wozniakowski,et al.  Iterative refinement implies numerical stability , 1977 .

[14]  J. Bunch,et al.  Rank-one modification of the symmetric eigenproblem , 1978 .

[15]  J. H. Wilkinson,et al.  AN ESTIMATE FOR THE CONDITION NUMBER OF A MATRIX , 1979 .

[16]  Charles L. Lawson,et al.  Basic Linear Algebra Subprograms for Fortran Usage , 1979, TOMS.

[17]  J. Cuppen.  A divide and conquer method for the symmetric tridiagonal eigenproblem , 1980 .

[18]  R. Skeel.  Iterative refinement implies numerical stability for Gaussian elimination , 1980 .

[19]  J. D. Roberts,et al.  Linear model reduction and solution of the algebraic Riccati equation by use of the sign function , 1980 .

[20]  Cleve B. Moler.  Demonstration of a matrix Laboratory , 1982 .

[21]  J. L. Howland.  The sign matrix and the separation of matrix eigenvalues , 1983 .

[22]  Gene H. Golub,et al.  Matrix computations , 1983 .

[23]  F. Gustavson,et al.  Implementing Linear Algebra Algorithms for Dense Matrices on a Vector Pipeline Machine , 1984 .

[24]  Christian H. Bischof,et al.  The WY representation for products of householder matrices , 1985, PPSC.

[25]  Danny C. Sorensen,et al.  Analysis of Pairwise Pivoting in Gaussian Elimination , 1985, IEEE Transactions on Computers.

[26]  L. Foster.  Rank and null space calculations using matrix decomposition without column interchanges , 1986 .

[27]  Jack Dongarra,et al.  LINPACK Users' Guide , 1987 .

[28]  T. Chan.  Rank revealing QR factorizations , 1987 .

[29]  V. Hari,et al.  On Jacobi methods for singular value decompositions , 1987 .

[30]  J. Barlow,et al.  Computing accurate eigensystems of scaled diagonally dominant matrices: LAPACK working note No. 7 , 1988 .

[31]  B. Parlett,et al.  Block reflectors: theory and computation , 1988 .

[32]  Jack J. Dongarra,et al.  Algorithm 656: an extended set of basic linear algebra subprograms: model implementation and test programs , 1988, TOMS.

[33]  Nicholas J. Higham,et al.  FORTRAN codes for estimating the one-norm of a real or complex matrix, with applications to condition estimation , 1988, TOMS.

[34]  J. Ortega.  Introduction to Parallel and Vector Solution of Linear Systems , 1988, Frontiers of Computer Science.

[35]  Jack J. Dongarra,et al.  An extended set of FORTRAN basic linear algebra subprograms , 1988, TOMS.

[36]  James Demmel,et al.  On a Block Implementation of Hessenberg Multishift QR Iteration , 1989, Int. J. High Speed Comput..

[37]  Stephen A. Vavasis.  Gaussian Elimination with Pivoting is P-Complete , 1989, SIAM J. Discret. Math..

[38]  C. Loan,et al.  A Storage-Efficient $WY$ Representation for Products of Householder Transformations , 1989 .

[39]  R. Schreiber,et al.  On the convergence of the cyclic Jacobi method for parallel block orderings , 1989 .

[40]  N. Higham,et al.  Large growth factors in Gaussian elimination with pivoting , 1989 .

[41]  P. P. Rijk.  A one-sided Jacobi algorithm for computing the singular value decomposition on avector computer , 1989 .

[42]  J. Demmel,et al.  The bidiagonal singular value decomposition and Hamiltonian mechanics: LAPACK working note No. 11 , 1989 .

[43]  I. Duff,et al.  On the augmented system approach to sparse least-squares problems , 1989 .

[44]  Jack J. Dongarra,et al.  A set of level 3 basic linear algebra subprograms , 1990, TOMS.

[45]  L. Trefethen,et al.  Average-case stability of Gaussian elimination , 1990 .

[46]  S. Batterson.  Convergence of the shifted QR algorithm on 3×3 normal matrices , 1990 .

[47]  K. A. Gallivan,et al.  Parallel Algorithms for Dense Linear Algebra Computations , 1990, SIAM Rev..

[48]  James Demmel,et al.  Accurate Singular Values of Bidiagonal Matrices , 1990, SIAM J. Sci. Comput..

[49]  Nicholas J. Higham,et al.  Exploiting fast matrix multiplication within the level 3 BLAS , 1990, TOMS.

[50]  D. Sorensen,et al.  Block reduction of matrices to condensed forms for eigenvalue computations , 1990 .

[51]  Jack J. Dongarra,et al.  Algorithm 679: A set of level 3 basic linear algebra subprograms: model implementation and test programs , 1990, TOMS.

[52]  Jack J. Dongarra,et al.  Solving linear systems on vector and shared memory computers , 1990 .

[53]  E. Jessup.  A case against a divide and conquer approach to the nonsymmetric eigenvalue problem , 1993 .

[54]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[55]  N. Gould.  On growth in Gaussian elimination with complete pivoting , 1991 .

[56]  E. Stickel.  Separating eigenvalues using the matrix sign function , 1991 .

[57]  Victor Y. Pan,et al.  An Improved Newton Iteration for the Generalized Inverse of a Matrix, with Applications , 1991, SIAM J. Sci. Comput..

[58]  Jack Dongarra,et al.  LAPACK Working Note 39: On Designing Portable High Performance Numerical Libraries , 1991 .

[59]  J. Demmel.  Open problems in numerical linear algebra , 1992 .

[60]  T. Y. Li,et al.  Solving eigenvalue problems of real nonsymmetric matrices with real homotopies , 1992 .

[61]  James Demmel,et al.  Stability of block algorithms with fast level-3 BLAS , 1992, TOMS.

[62]  A. Edelman.  The Complete Pivoting Conjecture for Gaussian Elimination is False , 1992 .

[63]  Bart De Moor,et al.  Generalizations of the Singular Value and QR-Decompositions , 1992, SIAM J. Matrix Anal. Appl..

[64]  C. Pan,et al.  Rank-Revealing QR Factorizations and the Singular Value Decomposition , 1992 .

[65]  Christopher C. Paige,et al.  Loss and Recapture of Orthogonality in the Modified Gram-Schmidt Algorithm , 1992, SIAM J. Matrix Anal. Appl..

[66]  J. Demmel.  Trading Off Parallelism and Numerical Stability , 1992 .

[67]  James Demmel,et al.  Jacobi's Method is More Accurate than QR , 1989, SIAM J. Matrix Anal. Appl..

[68]  G. W. Stewart,et al.  An updating algorithm for subspace tracking , 1992, IEEE Trans. Signal Process..

[69]  R Jessup,et al.  A Parallel Algorithm for Computing the Singular Value Decomposition of a Matrix:A Revision of Argonne National Laboratory Tech. Report ANL/MCS-TM-102 ; CU-CS-623-92 , 1994 .

[70]  John R. Gilbert,et al.  Sparse Matrices in MATLAB: Design and Implementation , 1992, SIAM J. Matrix Anal. Appl..

[71]  N. Higham,et al.  Stability of methods for matrix inversion , 1992 .

[72]  Ronald R. Coifman,et al.  Wavelet-Like Bases for the Fast Solution of Second-Kind Integral Equations , 1993, SIAM J. Sci. Comput..

[73]  Shing-Tung Yau,et al.  Reducing the Symmetric Matrix Eigenvalue Problem to Matrix Multiplications , 1993, SIAM J. Sci. Comput..

[74]  J. Demmel,et al.  On swapping diagonal blocks in real Schur form , 1993 .

[75]  James Demmel,et al.  On computing condition numbers for the nonsymmetric eigenproblem , 1993, TOMS.

[76]  James Demmel,et al.  Parallel numerical linear algebra , 1993, Acta Numerica.

[77]  Ivan Slapničar,et al.  Accurate Symmetric Eigenreduction by a Jacobi Method , 1993 .

[78]  James Demmel,et al.  Design of a Parallel Nonsymmetric Eigenroutine Toolbox, Part I , 1993, PPSC.

[79]  G. Stewart.  Updating a Rank-Revealing ULV Decomposition , 1993, SIAM J. Matrix Anal. Appl..

[80]  James Demmel,et al.  Computing the Generalized Singular Value Decomposition , 1993, SIAM J. Sci. Comput..

[81]  Stephen J. Wright.  A Collection of Problems for Which Gaussian Elimination with Partial Pivoting is Unstable , 1993, SIAM J. Sci. Comput..

[82]  Jack J. Dongarra,et al.  A Parallel Algorithm for the Nonsymmetric Eigenvalue Problem , 1991, SIAM J. Sci. Comput..

[83]  Ilse C. F. Ipsen,et al.  On Rank-Revealing Factorisations , 1994, SIAM J. Matrix Anal. Appl..

[84]  L. Foster.  Gaussian Elimination with Partial Pivoting Can Fail in Practice , 1994, SIAM J. Matrix Anal. Appl..

[85]  S. Eisenstat,et al.  A Stable and Efficient Algorithm for the Rank-One Modification of the Symmetric Eigenproblem , 1994, SIAM J. Matrix Anal. Appl..

[86]  Å. Björck,et al.  Solution of augmented linear systems using orthogonal factorizations , 1994 .

[87]  Mark T. Jones,et al.  Factoring Symmetric Indefinite Matrices on High-Performance Architectures , 1994 .

[88]  J. Demmel,et al.  LAPACK Working Note 88: Efficient Computation of the Singular Value Decomposition with Applications to Least Squares Problems , 1994 .

[89]  J. Demmel,et al.  Inverse Free Parallel Spectral Divide and Conquer Algorithms for Nonsymmetric Eigenproblems , 1994 .

[90]  B. Parlett,et al.  Accurate singular values and differential qd algorithms , 1994 .

[91]  J. Demmel,et al.  On the correctness of some bisection-like parallel eigenvalue algorithms in floating point arithmetic. , 1995 .

[92]  Beresford N. Parlett,et al.  The New qd Algorithms , 1995, Acta Numerica.

[93]  Linda Kaufman.  Computing the MDMT decomposition , 1995, TOMS.

[94]  Stanley C. Eisenstat,et al.  A Divide-and-Conquer Algorithm for the Bidiagonal SVD , 1995, SIAM J. Matrix Anal. Appl..

[95]  Jack Dongarra,et al.  LAPACK Working Note 109 BLAS Technical Workshop , 1995 .

[96]  Roy Mathias.  Accurate Eigensystem Computations by Jacobi Methods , 1995, SIAM J. Matrix Anal. Appl..

[97]  Ji-Guang Sun,et al.  Optimal backward perturbation bounds for the linear least squares problem , 1995, Numer. Linear Algebra Appl..

[98]  James R. Bunch,et al.  Bounding the Subspaces from Rank Revealing Two-Sided Orthogonal Decompositions , 1995, SIAM J. Matrix Anal. Appl..

[99]  James Demmel,et al.  Stability of block LU factorization , 1992, Numer. Linear Algebra Appl..

[100]  Beresford N. Parlett,et al.  Implicit Cholesky algorithms for singular values and vectors of triangular matrices , 1993, Numer. Linear Algebra Appl..

[101]  M. SIAMJ..  STABILITY OF THE DIAGONAL PIVOTING METHOD WITH PARTIAL PIVOTING , 1995 .

[102]  A. Laub,et al.  The matrix sign function , 1995, IEEE Trans. Autom. Control..

[103]  Nicholas J. Higham.  Stability of Parallel Triangular System Solvers , 1995, SIAM J. Sci. Comput..

[104]  Jack Dongarra,et al.  LAPACK++ V. 1.0: High Performance Linear Algebra Users'' Guides , 1995 .

[105]  Jack Dongarra,et al.  Libraries for linear algebra , 1995 .

[106]  Haesun Park,et al.  Downdating the Rank-Revealing URV Decomposition , 1995, SIAM J. Matrix Anal. Appl..

[107]  Ming Gu,et al.  Efficient Algorithms for Computing a Strong Rank-Revealing QR Factorization , 1996, SIAM J. Sci. Comput..

[108]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[109]  Gisela Engeln-Müllges,et al.  Direct Methods for Solving Systems of Linear Equations , 1996 .

[110]  H. Zha,et al.  An algorithm and a stability theory for downdating the ULV decomposition , 1996 .

[111]  Ji-guang Sun.  Optimal backward perturbation bounds for the linear least-squares problem with multiple right-hand sides , 1996 .

[112]  Robert A. van de Geijn,et al.  Parallelizing the QR Algorithm for the Unsymmetric Algebraic Eigenvalue Problem: Myths and Reality , 1996, SIAM J. Sci. Comput..

[113]  D. Day.  How the QR algorithm fails to converge and how to fix it , 1996 .

[114]  Jaeyoung Choi,et al.  Design and Implementation of the ScaLAPACK LU, QR, and Cholesky Factorization Routines , 1994, Sci. Program..

[115]  Ji-guang Sun,et al.  Optimal Backward Perturbation Bounds for Underdetermined Systems , 1997 .

[116]  T. Chan,et al.  Probabilistic Analysis of Gaussian Elimination Without Pivoting , 1997 .

[117]  John G. Lewis,et al.  Accurate Symmetric Indefinite Linear Equation Solvers , 1999, SIAM J. Matrix Anal. Appl..

[118]  J. Barlow,et al.  An efficient rank detection procedure for modifying the ULV decomposition , 1998 .

[119]  A. Bjijrck.  Stability Analysis of the Method of Seminormal Equations for Linear Least Squares Problems , 2001 .

[120]  David H. Bailey,et al.  Using Strassen's algorithm to accelerate the solution of linear systems , 1991, The Journal of Supercomputing.

[121]  Bo Kågström,et al.  Computing eigenspaces with specified eigenvalues of a regular matrix pair (A, B) and condition estimation: theory, algorithms and software , 1996, Numerical Algorithms.

[122]  Y. Danieli.  Guide , 2005 .

[123]  Gene H. Golub,et al.  Some modified matrix eigenvalue problems , 1973, Milestones in Matrix Computation.