A matrix analysis of conjugate gradient algorithms

This paper explores the relationships between the conjugate gradient algorithms Orthodir, Orthomin, and Orthores. To facilitate this exploration, a matrix formulation for each algorithm is given. It is shown that Orthodir directly computes a Hessenberg matrix H{sub k} at step k. Orthores also computes a Hessenberg matrix, G{sub k}, which is similar to a Hessenberg matrix obtained from H{sub k} by perturbing its last column. (This perturbation vanishes at convergence.) Orthomin, on the other hand, computes a UL and LU factorization of the perturbed H{sub k} and G{sub k}, respectively. The breakdown of Orthomin and Orthores are interpreted in terms of these underlying matrix factorizations. A connection with Lanczos algorithms is also examined, as is the special case of B-normal(1) matrices (for which efficient three-term CG algorithms exist).