Numerical evaluation of the Communication-Avoiding Lanczos algorithm

The Lanczos algorithm is widely used for solving large sparse symmetric eigenvalue problems when only a few eigenvalues from the spectrum are needed. Due to sparse matrix-vector multiplications and frequent synchronization, the algorithm is communication intensive leading to poor performance on parallel computers and modern cache-based processors. The Communication-Avoiding Lanczos algorithm [Hoemmen; 2010] attempts to improve performance by taking the equivalence of s steps of the original algorithm at a time. The scheme is equivalent to the original algorithm in exact arithmetic but as the value of s grows larger, numerical roundo errors are expected to have a greater impact. In this paper, we investigate the numerical properties of the Communication-Avoiding Lanczos (CA-Lanczos) algorithm and how well it works in practical computations. Apart from the algorithm itself, we have implemented techniques that are commonly used with the Lanczos algorithm to improve its numerical performance, such as semi-orthogonal schemes and restarting. We present results that show that CA-Lanczos is often as accurate as the original algorithm. In many cases, if the parameters of the s-step basis are chosen appropriately, the numerical behaviour of CA-Lanczos is close to the standard algorithm even though it is somewhat more sensitive to loosing mutual orthogonality among the basis vectors.

[1]  Sverker Holmgren,et al.  Communication-Efficient Algorithms for Numerical Quantum Dynamics , 2010, PARA.

[2]  Mark Hoemmen,et al.  Communication-avoiding Krylov subspace methods , 2010 .

[3]  H. Simon.  The Lanczos algorithm with partial reorthogonalization , 1984 .

[4]  C. Lanczos.  An iteration method for the solution of the eigenvalue problem of linear differential and integral operators , 1950 .

[5]  G. W. Stewart.  Block Gram--Schmidt Orthogonalization , 2008, SIAM J. Sci. Comput..

[6]  J. Grcar.  Analyses of the lanczos algorithm and of the approximation problem in richardson's method , 1981 .

[7]  James Demmel,et al.  Minimizing communication in sparse matrix solvers , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[8]  Anthony T. Chronopoulos,et al.  s-step iterative methods for symmetric linear systems , 1989 .

[9]  Mark Hoemmen,et al.  A Communication-Avoiding, Hybrid-Parallel, Rank-Revealing Orthogonalization Method , 2011, 2011 IEEE International Parallel & Distributed Processing Symposium.

[10]  C. Paige.  Computational variants of the Lanczos method for the eigenproblem , 1972 .

[11]  Kesheng Wu,et al.  Thick-Restart Lanczos Method for Large Symmetric Eigenvalue Problems , 2000, SIAM J. Matrix Anal. Appl..

[12]  C. Paige.  Error Analysis of the Lanczos Algorithm for Tridiagonalizing a Symmetric Matrix , 1976 .

[13]  James Demmel,et al.  Avoiding Communication in Two-Sided Krylov Subspace Methods , 2011 .

[14]  W. Karush.  An iterative method for finding characteristic vectors of a symmetric matrix , 1951 .

[15]  Jack Dongarra,et al.  Templates for the Solution of Algebraic Eigenvalue Problems , 2000, Software, environments, tools.

[16]  B. Parlett,et al.  The Lanczos algorithm with selective orthogonalization , 1979 .

[17]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[18]  Anthony T. Chronopoulos,et al.  A class of Lanczos-like algorithms implemented on parallel computers , 1991, Parallel Comput..

[19]  Andrés Tomás,et al.  Evaluation of Several Variants of Explicitly Restarted Lanczos Eigensolvers and Their Parallel Implementations , 2006, VECPAR.

[20]  Maurice Clint,et al.  Explicitly restarted Lanczos algorithms in an MPP environment , 1999, Parallel Comput..

[21]  H. Simon.  Analysis of the symmetric Lanczos algorithm with reorthogonalization methods , 1984 .

[22]  B. Gellai.  The Lanczos Method , 2010 .

[23]  D. Calvetti,et al.  AN IMPLICITLY RESTARTED LANCZOS METHOD FOR LARGE SYMMETRIC EIGENVALUE PROBLEMS , 1994 .