The Effects of Loss of Orthogonality on Large Scale Numerical Computations

Many useful large sparse matrix algorithms are based on orthogonality, but for efficiency this orthogonality is often obtained via short term recurrences. This can lead to both loss of orthogonality and loss of linear independence of computed vectors, yet with well designed algorithms high accuracy can still be obtained. Here we discuss a nice theoretical indicator of loss of orthogonality and linear independence and show how it leads to a related higher dimensional orthogonality that can be used to analyze and prove the effectiveness of such algorithms. We illustrate advantages and shortcomings of such algorithms with Cornelius Lanczos’ Hermitian matrix tridiagonalization process. The paper is reasonably expository, keeping simple by avoiding some detailed analyses.

[1]  CHRISTOPHER C. PAIGE,et al.  Accuracy of the Lanczos Process for the Eigenproblem and Solution of Equations , 2019, SIAM J. Matrix Anal. Appl..

[2]  C. Lanczos.  An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[3]  G. Stewart.  On the Perturbation of Pseudo-Inverses, Projections and Linear Least Squares Problems , 1977 .

[4]  Christopher C. Paige,et al.  Properties of a Unitary Matrix Obtained from a Sequence of Normalized Vectors , 2014, SIAM J. Matrix Anal. Appl..

[5]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[6]  C. Paige.  Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem , 1980 .

[7]  Gene H. Golub,et al.  Calculating the singular values and pseudo-inverse of a matrix , 2007, Milestones in Matrix Computation.

[8]  Christopher C. Paige,et al.  An Augmented Stability Result for the Lanczos Hermitian Matrix Tridiagonalization Process , 2010, SIAM J. Matrix Anal. Appl..

[9]  Christopher C. Paige,et al.  Loss and Recapture of Orthogonality in the Modified Gram-Schmidt Algorithm , 1992, SIAM J. Matrix Anal. Appl..

[10]  Christopher C. Paige,et al.  A Useful Form of Unitary Matrix Obtained from Any Sequence of Unit 2-Norm n-Vectors , 2009, SIAM J. Matrix Anal. Appl..

[11]  Zdenek Strakos,et al.  Core Problems in Linear Algebraic Systems , 2005, SIAM J. Matrix Anal. Appl..

[12]  C. Lanczos.  Solution of Systems of Linear Equations by Minimized Iterations1 , 1952 .

[13]  Michael A. Saunders,et al.  LSMR: An Iterative Algorithm for Sparse Least-Squares Problems , 2011, SIAM J. Sci. Comput..

[14]  William Kahan,et al.  Some new bounds on perturbation of subspaces , 1969 .

[15]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[16]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[17]  Miroslav Rozlozník,et al.  Modified Gram-Schmidt (MGS), Least Squares, and Backward Stability of MGS-GMRES , 2006, SIAM J. Matrix Anal. Appl..

[18]  Christopher C. Paige,et al.  Scaled total least squares fundamentals , 2002, Numerische Mathematik.