A cooperative conjugate gradient method for linear systems permitting multithread implementation of low complexity

This paper proposes a generalization of the conjugate gradient (CG) method used to solve the equation Ax = b for a symmetric positive definite matrix A of large size n. The generalization consists of permitting the scalar control parameters (= stepsizes in gradient and conjugate gradient directions) to be replaced by matrices, so that multiple descent and conjugate directions are updated simultaneously. Implementation involves the use of multiple agents or threads and is referred to as cooperative CG (cCG), in which the cooperation between agents resides in the fact that the calculation of each entry of the control parameter matrix now involves information that comes from the other agents. The multithread implementation is shown to have low worst case complexity equation in exact arithmetic. Numerical experiments, that illustrate the interest of theoretical results, are carried out on a multicore computer.

[1]  D. O’Leary.  The block conjugate gradient algorithm and related methods , 1980 .

[2]  Henryk Wozniakowski,et al.  On the Optimal Solution of Large Linear Systems , 1984, JACM.

[3]  Anne Greenbaum,et al.  Iterative methods for solving linear systems , 1997, Frontiers in applied mathematics.

[4]  Audra E. Kosh,et al.  Linear Algebra and its Applications , 1992 .

[5]  Convex Optimization in Signal Processing and Communications , 2010 .

[6]  Claude Brezinski,et al.  Multiparameter descent methods , 1999 .

[7]  G. Meurant,et al.  The Lanczos and conjugate gradient algorithms in finite precision arithmetic , 2006, Acta Numerica.

[8]  Paul Tseng.  Control perspectives on numerical algorithms and matrix problems , 2008, Math. Comput..

[9]  Douglass J. Wilde,et al.  Foundations of Optimization. , 1967 .

[10]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[11]  C. Brezinski,et al.  Hybrid procedures for solving linear systems , 1994 .

[12]  Richard M. Murray,et al.  Recent Research in Cooperative Control of Multivehicle Systems , 2007 .

[13]  Anne Greenbaum.  11. Incomplete Decompositions , 1997 .

[14]  Pierre-Alexandre Bliman,et al.  Cooperative parallel asynchronous computation of the solution of symmetric linear systems , 2010, 49th IEEE Conference on Decision and Control (CDC).

[15]  Eugenius Kaszkurewicz,et al.  Control Perspectives on Numerical Algorithms And Matrix Problems (Advances in Design and Control) (Advances in Design and Control 10) , 2006 .