Empirical Study for Communication Cost of Parallel Conjugate Gradient on a Star-Based Network

Conjugate Gradient is an iterative linear solver that is used in many scientific and engineering applications to solve a system of linear equations. However, Conjugate Gradient generates a heavy load of computation and therefore it slows the performance of the applications using it. In this paper, we conduct an empirical cost study of a parallel CG on our star-based network. We evaluate the communication overhead involved by a parallel CG. In particular, we derive network parameters; the Maximum Transfer Unit (MTU), that can contribute to the optimization of communication cost and to the reduction of the waiting overhead of the parallel algorithm.

[1]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[2]  Susumu Yamada,et al.  High Performance Computing for Eigenvalue Solver in Density-Matrix Renormalization Group Method: Parallelization of the Hamiltonian Matrix-Vector Multiplication , 2008, VECPAR.

[3]  Youngjoo Kim,et al.  Performance of Parallel Conjugate Gradient Solvers in Meshfree Analysis , 2003 .

[4]  Andrzej Jordan,et al.  A new version of conjugate gradient method parallel implementation , 2002, Proceedings. International Conference on Parallel Computing in Electrical Engineering.

[5]  J. Shewchuk.  An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[6]  D. O’Leary.  The block conjugate gradient algorithm and related methods , 1980 .

[7]  J.G. Lewis,et al.  Matrix-vector multiplication and conjugate gradient algorithms on distributed memory computers , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[8]  M. Møller.  A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1990 .

[9]  J. G. Lewis,et al.  Distributed memory matrix-vector multiplication and conjugate gradient algorithms , 1993, Supercomputing '93.

[10]  Gerd Heber,et al.  Landing CG on EARTH: A Case Study of Fine-Grained Multithreading on an Evolutionary Path , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[11]  Martyn R. Field.  Optimizing a Parallel Conjugate Gradient Solver , 1998, SIAM J. Sci. Comput..

[12]  Andrzej Jordan,et al.  The Parallel Algorithm of Conjugate Gradient Method , 2001, IWCC.

[13]  Guang R. Gao,et al.  Implementing parallel conjugate gradient on the EARTH multithreaded architecture , 2004, 2004 IEEE International Conference on Cluster Computing (IEEE Cat. No.04EX935).

[14]  Dianne P. O'Leary,et al.  Parallel implementation of the block conjugate gradient algorithm , 1987, Parallel Comput..

[15]  Hee-Dae Kwon.  Efficient parallel implementations of finite element methods based on the conjugate gradient method , 2003, Appl. Math. Comput..