Jacobi Correction Equation, Line Search, and Conjugate Gradients in Hermitian Eigenvalue Computation I: Computing an Extreme Eigenvalue

This paper is concerned with the convergence properties of iterative algorithms of conjugate gradient type applied to the computation of an extreme eigenvalue of a Hermitian operator via the optimization of the Rayleigh quotient functional. The algorithms in focus employ the line search for the extremum of the Rayleigh quotient functional in the direction that is a linear combination of the gradient and the previous search direction. An asymptotically explicit equation for the reduction in the eigenvalue error after the line search step as a function of the search direction is derived and applied to the analysis of the local convergence properties (i.e., the error reduction after an iteration) of the algorithms in question. The local (stepwise) asymptotic equivalence of various conjugate gradient algorithms and the generalized Davidson method is proved, and a new convergence estimate for conjugate gradient iterations is derived, showing the reduction in the eigenvalue error after any two consecutive iterations. The paper's analysis extensively employs remarkable properties of the operator of the so-called Jacobi orthogonal complement correction equation.

[1]  G. Gambolati,et al.  Extreme eigenvalues of large sparse matrices by Rayleigh quotient and modified conjugate gradients , 1986 .

[2]  A. Sameh,et al.  The trace minimization method for the symmetric generalized eigenvalue problem , 2000 .

[3]  L. Eldén,et al.  Inexact Rayleigh Quotient-Type Methods for Eigenvalue Computations , 2002 .

[4]  Merico E. Argentati,et al.  Block Locally Optimal Preconditioned Eigenvalue Xolvers (BLOPEX) in hypre and PETSc , 2007, SIAM J. Sci. Comput..

[5]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[6]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[7]  Y. Feng,et al.  CONJUGATE GRADIENT METHODS FOR SOLVING THE SMALLEST EIGENPAIR OF LARGE SYMMETRIC EIGENVALUE PROBLEMS , 1996 .

[8]  E. Ovtchinnikov.  Cluster robust error estimates for the Rayleigh–Ritz approximation II: Estimates for eigenvalues , 2006 .

[9]  Evgueni E. Ovtchinnikov,et al.  Convergence Estimates for the Generalized Davidson Method for Symmetric Eigenvalue Problems I: The Preconditioning Aspect , 2003, SIAM J. Numer. Anal..

[10]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[11]  J. Daniel.  The Conjugate Gradient Method for Linear and Nonlinear Operator Equations , 1967 .

[12]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[13]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[14]  Yvan Notay,et al.  Combination of Jacobi–Davidson and conjugate gradients for the partial symmetric eigenproblem , 2002, Numer. Linear Algebra Appl..

[15]  Evgueni E. Ovtchinnikov,et al.  Convergence Estimates for the Generalized Davidson Method for Symmetric Eigenvalue Problems II: The Subspace Acceleration , 2003, SIAM J. Numer. Anal..

[16]  J. Brandts.  The Riccati algorithm for eigenvalues and invariant subspaces of matrices with inexpensive action , 2003 .

[17]  P. Absil,et al.  A truncated-CG style method for symmetric generalized eigenvalue problems , 2006 .

[18]  Andrew V. Knyazev,et al.  Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method , 2001, SIAM J. Sci. Comput..

[19]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[20]  R. Fletcher,et al.  New iterative methods for solution of the eigenproblem , 1966 .

[21]  Tapan K. Sarkar,et al.  A survey of conjugate gradient algorithms for solution of extreme eigen-problems of a symmetric matrix , 1989, IEEE Trans. Acoust. Speech Signal Process..

[22]  Yvan Notay,et al.  Convergence Analysis of Inexact Rayleigh Quotient Iteration , 2002, SIAM J. Matrix Anal. Appl..

[23]  Yunkai Zhou,et al.  Studies on Jacobi–Davidson, Rayleigh quotient iteration, inverse iteration generalized Davidson and Newton updates , 2006, Numer. Linear Algebra Appl..

[24]  E. Ovtchinnikov.  Cluster robust error estimates for the Rayleigh–Ritz approximation I: Estimates for invariant subspaces , 2006 .

[25]  Andrew Knyazev,et al.  Preconditioned Eigensolvers - an Oxymoron? , 1998 .

[26]  Evgueni E. Ovtchinnikov.  Sharp Convergence Estimates for the Preconditioned Steepest Descent Method for Hermitian Eigenvalue Problems , 2006, SIAM J. Numer. Anal..

[27]  Klaus Neymeyr,et al.  A geometric theory for preconditioned inverse iteration applied to a subspace , 2002, Math. Comput..

[28]  A. Knyazev.  A Preconditioned Conjugate Gradient Method for Eigenvalue Problems and its Implementation in a Subspace , 1991 .

[29]  E. Polak,et al.  Computational methods in optimization : a unified approach , 1972 .

[30]  U. Hetmaniuk,et al.  A comparison of eigensolvers for large‐scale 3D modal analysis using AMG‐preconditioned iterative methods , 2005 .

[31]  K. Neymeyr.  A geometric theory for preconditioned inverse iteration. I : Extrema of the Rayleigh quotient , 2001 .

[32]  Evgueni E. Ovtchinnikov.  Jacobi Correction Equation, Line Search, and Conjugate Gradients in Hermitian Eigenvalue Computation II: Computing Several Extreme Eigenvalues , 2008, SIAM J. Numer. Anal..

[33]  A. Knyazev.  Convergence rate estimates for iterative methods for a mesh symmetrie eigenvalue problem , 1987 .

[34]  E. M. Dowling,et al.  Conjugate gradient eigenstructure tracking for adaptive spectral estimation , 1995, IEEE Trans. Signal Process..

[35]  Gerard L. G. Sleijpen,et al.  A Jacobi-Davidson Iteration Method for Linear Eigenvalue Problems , 1996, SIAM J. Matrix Anal. Appl..