Accuracy of the s-Step Lanczos Method for the Symmetric Eigenproblem in Finite Precision

The $s$-step Lanczos method can achieve an $O(s)$ reduction in data movement over the classical Lanczos method for a fixed number of iterations, allowing the potential for significant speedups on modern computers. However, although the $s$-step Lanczos method is equivalent to the classical Lanczos method in exact arithmetic, it can behave quite differently in finite precision. Increased roundoff errors can manifest as a loss of accuracy or deterioration of convergence relative to the classical method, reducing the potential performance benefits of the $s$-step approach. In this paper, we present for the first time a complete rounding error analysis of the $s$-step Lanczos method. Our methodology is analogous to Paige's rounding error analysis for classical Lanczos [IMA J. Appl. Math., 18 (1976), pp. 341--349]. Our analysis gives upper bounds on the loss of normality of and orthogonality between the computed Lanczos vectors, as well as a recurrence for the loss of orthogonality. We further demonstrate that...

[1]  Anthony T. Chronopoulos,et al.  s-step iterative methods for symmetric linear systems , 1989 .

[2]  Lothar Reichel,et al.  On the generation of Krylov subspace bases , 2012 .

[3]  Claude Brezinski,et al.  Lanczos Tridiagonalization Process , 2002 .

[4]  James Demmel,et al.  Minimizing communication in sparse matrix solvers , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[5]  John Van Rosendale.  Minimizing Inner Product Data Dependencies in Conjugate Gradient Iteration , 1983, ICPP.

[6]  James Demmel,et al.  Avoiding communication in sparse matrix computations , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[7]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[8]  Anne Greenbaum,et al.  Predicting the Behavior of Finite Precision Lanczos and Conjugate Gradient Computations , 2015, SIAM J. Matrix Anal. Appl..

[9]  C. Paige.  Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem , 1980 .

[10]  H. Simon.  The Lanczos algorithm with partial reorthogonalization , 1984 .

[11]  Marghoob Mohiyuddin,et al.  Tuning Hardware and Software for Multiprocessors , 2012 .

[12]  Anthony T. Chronopoulos,et al.  An efficient nonsymmetric Lanczos method on parallel vector computers , 1992 .

[13]  H. Walker.  Implementation of the GMRES method using householder transformations , 1988 .

[14]  C. Paige.  Error Analysis of the Lanczos Algorithm for Tridiagonalizing a Symmetric Matrix , 1976 .

[15]  Wolfgang Wülling.  On Stabilization and Convergence of Clustered Ritz Values in the Lanczos Method , 2005, SIAM J. Matrix Anal. Appl..

[16]  Siegfried M. Rump,et al.  Verified bounds for singular values, in particular for the spectral norm of a matrix and its inverse , 2011 .

[17]  W. Joubert,et al.  Parallelizable restarted iterative methods for nonsymmetric linear systems. part I: Theory , 1992 .

[18]  Martin H. Gutknecht,et al.  Lanczos-type solvers for nonsymmetric linear systems of equations , 1997, Acta Numerica.

[19]  J. Demmel,et al.  Avoiding Communication in Computing Krylov Subspaces , 2007 .

[20]  Anthony T. Chronopoulos,et al.  A class of Lanczos-like algorithms implemented on parallel computers , 1991, Parallel Comput..

[21]  Christopher C. Paige,et al.  The computation of eigenvalues and eigenvectors of very large sparse matrices , 1971 .

[22]  D. Hut.  A Newton Basis Gmres Implementation , 1991 .

[23]  G. Meurant.  The Lanczos and Conjugate Gradient Algorithms: From Theory to Finite Precision Computations , 2006 .

[24]  G. Meurant,et al.  The Lanczos and conjugate gradient algorithms in finite precision arithmetic , 2006, Acta Numerica.

[25]  C. Paige.  Computational variants of the Lanczos method for the eigenproblem , 1972 .

[26]  James Demmel,et al.  A Residual Replacement Strategy for Improving the Maximum Attainable Accuracy of s-Step Krylov Subspace Methods , 2014, SIAM J. Matrix Anal. Appl..

[27]  C. Lanczos.  An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[28]  Mark Hoemmen,et al.  Communication-avoiding Krylov subspace methods , 2010 .

[29]  Samuel Williams,et al.  s-Step Krylov Subspace Methods as Bottom Solvers for Geometric Multigrid , 2014, 2014 IEEE 28th International Parallel and Distributed Processing Symposium.

[30]  James Demmel,et al.  Avoiding Communication in Nonsymmetric Lanczos-Based Krylov Subspace Methods , 2013, SIAM J. Sci. Comput..

[31]  B. Parlett,et al.  The Lanczos algorithm with selective orthogonalization , 1979 .

[32]  Jens-Peter M. Zemke,et al.  Krylov Subspace Methods in Finite Precision : A Unified Approach , 2003 .

[33]  R. C. Thompson,et al.  Principal submatrices II: the upper and lower quadratic inequalities☆ , 1968 .

[34]  Anthony T. Chronopoulos,et al.  On the efficient implementation of preconditioned s-step conjugate gradient methods on multiprocessors with memory hierarchy , 1989, Parallel Comput..

[35]  Graham F. Carey,et al.  Parallelizable Restarted Iterative Methods for Nonsymmetric Linear Systems , 1991, PPSC.

[36]  Dennis Gannon,et al.  On the Impact of Communication Complexity on the Design of Parallel Numerical Algorithms , 1984, IEEE Transactions on Computers.

[37]  Eric de Sturler,et al.  A Performance Model for Krylov Subspace Methods on Mesh-Based Parallel Computers , 1996, Parallel Comput..

[38]  James Demmel,et al.  Error Analysis of the S-Step Lanczos Method in Finite Precision , 2014 .

[39]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[40]  Qiang Ye,et al.  Residual Replacement Strategies for Krylov Subspace Iterative Methods for the Convergence of True Residuals , 2000, SIAM J. Sci. Comput..

[41]  Anthony T. Chronopoulos,et al.  Parallel Iterative S-Step Methods for Unsymmetric Linear Systems , 1996, Parallel Comput..

[42]  James Demmel,et al.  Analysis of the Finite Precision s-Step Biconjugate Gradient Method , 2014 .

[43]  Sivan Toledo,et al.  Quantitative performance modeling of scientific computations and creating locality in numerical algorithms , 1995 .

[44]  James Demmel,et al.  Communication lower bounds and optimal algorithms for numerical linear algebra*† , 2014, Acta Numerica.

[45]  H. Walker,et al.  Note on a Householder implementation of the GMRES method , 1986 .

[46]  Zdenek Strakos,et al.  Accuracy of Two Three-term and Three Two-term Recurrences for Krylov Space Solvers , 2000, SIAM J. Matrix Anal. Appl..

[47]  Christopher C. Paige,et al.  An Augmented Stability Result for the Lanczos Hermitian Matrix Tridiagonalization Process , 2010, SIAM J. Matrix Anal. Appl..