Implementation and performance evaluation of a distributed conjugate gradient method in a cloud computing environment

Cloud computing is an emerging technology where information technology resources are provisioned to users in a set of a unified computing resources on a pay per use basis. The resources are dynamically chosen to satisfy a user service level agreement and a required level of performance. A cloud is seen as a computing platform for heavy load applications. Conjugate gradient (CG) method is an iterative linear solver that is used by many scientific and engineering applications to solve a linear system of algebraic equations. CG generates a heavy load of computation, and therefore, it slows the performance of the applications using it. Distributing CG is considered as a way to increase its performance. However, running a distributed CG, based on a standard API, such as Message Passing Interface, in a cloud face many challenges, such as the cloud processing and networking capabilities. In this work, we present an in‐depth analysis of the CG algorithm and its complexity to develop adequate distributed algorithms. The implementation of these algorithms and their evaluation in our cloud environment reveal the gains and losses achieved by distributing the CG. The performance results show that despite the complexity of the CG processing and communication, a speedup gain of at least 1157.7 is obtained using 128 cores compared with National Aeronautics and Space Administration Advanced Supercomputing sequential execution. Given the emergence of clouds, the results in this paper analyzes performance issues when a generic public cloud, along with a standard development library, such as Message Passing Interface, is used for high‐performance applications, without the need of some specialized hardware and software. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  P. Aldo Moro.  Conjugate-Gradients Algorithms : An MPI-OpenMP Implementation on Distributed Shared Memory Systems , 1999 .

[2]  Youngjoo Kim,et al.  Performance of Parallel Conjugate Gradient Solvers in Meshfree Analysis , 2003 .

[3]  Rajkumar Buyya,et al.  Market-Oriented Cloud Computing: Vision, Hype, and Reality for Delivering IT Services as Computing Utilities , 2008, 2008 10th IEEE International Conference on High Performance Computing and Communications.

[4]  Gerd Heber,et al.  Developing a Communication Intensive Application on the EARTH Multithreaded Architecture (Distinguished Paper) , 2000, Euro-Par.

[5]  Martyn R. Field.  Optimizing a Parallel Conjugate Gradient Solver , 1998, SIAM J. Sci. Comput..

[6]  Mark Hoemmen,et al.  Communication-avoiding Krylov subspace methods , 2010 .

[7]  J. G. Lewis,et al.  Distributed memory matrix-vector multiplication and conjugate gradient algorithms , 1993, Supercomputing '93.

[8]  J. Shewchuk.  An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[9]  D. O’Leary.  The block conjugate gradient algorithm and related methods , 1980 .

[10]  Rajkumar Buyya,et al.  High-Performance Cloud Computing: A View of Scientific Applications , 2009, 2009 10th International Symposium on Pervasive Systems, Algorithms, and Networks.

[11]  J.G. Lewis,et al.  Matrix-vector multiplication and conjugate gradient algorithms on distributed memory computers , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[12]  池本 顕一,et al.  Service Level Agreementに基づいたサービス管理方法 , 1996 .

[13]  Guang R. Gao,et al.  Implementing parallel conjugate gradient on the EARTH multithreaded architecture , 2004, 2004 IEEE International Conference on Cluster Computing (IEEE Cat. No.04EX935).

[14]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[15]  Hee-Dae Kwon.  Efficient parallel implementations of finite element methods based on the conjugate gradient method , 2003, Appl. Math. Comput..

[16]  Khaled Shuaib,et al.  Empirical Study for Communication Cost of Parallel Conjugate Gradient on a Star-Based Network , 2010, 2010 Fourth Asia International Conference on Mathematical/Analytical Modelling and Computer Simulation.

[17]  Andrzej Jordan,et al.  The Parallel Algorithm of Conjugate Gradient Method , 2001, IWCC.

[18]  Dianne P. O'Leary,et al.  Parallel implementation of the block conjugate gradient algorithm , 1987, Parallel Comput..

[19]  Peter R. Jones,et al.  Implementation and Evaluation , 1995 .

[20]  Martin Fodslette Møller,et al.  A scaled conjugate gradient algorithm for fast supervised learning , 1993, Neural Networks.

[21]  David H. Bailey,et al.  The Nas Parallel Benchmarks , 1991, Int. J. High Perform. Comput. Appl..

[22]  Andrzej Jordan,et al.  A new version of conjugate gradient method parallel implementation , 2002, Proceedings. International Conference on Parallel Computing in Electrical Engineering.

[23]  Larry S.K. Fung,et al.  From Mega Cell to Giga Cell Reservoir Simulation , 2008 .

[24]  Gerd Heber,et al.  Implementation and evaluation of a communication intensive application on the EARTH multithreaded system , 2002, Concurr. Comput. Pract. Exp..

[25]  Randy H. Katz,et al.  Above the Clouds: A Berkeley View of Cloud Computing , 2009 .

[26]  Rajkumar Buyya,et al.  Article in Press Future Generation Computer Systems ( ) – Future Generation Computer Systems Cloud Computing and Emerging It Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility , 2022 .

[27]  Gerd Heber,et al.  Landing CG on EARTH: A Case Study of Fine-Grained Multithreading on an Evolutionary Path , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[28]  J. H. Abou-Kassem,et al.  Petroleum reservoir simulation : a basic approach , 2006 .

[29]  L. Ismail.  Communication issues in parallel Conjugate Gradient method using a star-based network , 2010, 2010 International Conference on Computer Applications and Industrial Electronics.