A review on accelerating scientific computations using the Conjugate Gradient method

Conjugate Gradient method is a very efficient iterative method for solving large systems of equations arising from real life scientific computing applications. In this paper we present the Conjugate Gradient method and its variants in brief. We also present a comparative analysis of implementations of this method on various platforms like FPGAs, GPUs etc which are suitable for High Performance Computing.

[1]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[2]  Xu Guo,et al.  Matrix-free GPU implementation of a preconditioned conjugate gradient solver for anisotropic elliptic PDEs , 2013, Comput. Vis. Sci..

[3]  Marcel van der Veen.  Sparse matrix vector multiplication on a field programmable gate array , 2007 .

[4]  Satoshi Matsuoka,et al.  Fast Conjugate Gradients with Multiple GPUs , 2009, ICCS.

[5]  I. C. Decker,et al.  Parallel implementation of a power system dynamic simulation methodology using the conjugate gradient method , 1991 .

[6]  M. J. D. Powell,et al.  Restart procedures for the conjugate gradient method , 1977, Math. Program..

[7]  Jing Hu,et al.  Solution of partial differential equations using reconfigurable computing , 2011 .

[8]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[9]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[10]  Steve Poole,et al.  An Implementation of the Conjugate Gradient Algorithm on FPGAs , 2008, 2008 16th International Symposium on Field-Programmable Custom Computing Machines.

[11]  Behzad Mahdavikhah Seyed Mehrabad.  A Multiple-FPGA Parallel Computing Architecture for Real-time Simulation of Deformable Objects , 2009 .

[12]  Viktor K. Prasanna,et al.  A Hybrid Approach for Mapping Conjugate Gradient onto an FPGA-Augmented Reconfigurable Supercomputer , 2006, 2006 14th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[13]  A. Sameh,et al.  The behavior of conjugate gradient algorithms on a multivector processor with a hierarchical memory , 1988 .

[14]  Sinan Shi,et al.  GPU Implementation of Iterative Solvers in Numerical Weather Predicting Models , 2013 .

[15]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[16]  Oleg Maslennikov,et al.  FPGA Implementation of the Conjugate Gradient Method , 2005, PPAM.

[17]  Shanq-Jang Ruan,et al.  Sparse matrix-vector multiplication on network-on-chip , 2010 .

[18]  Thomas Boorman,et al.  Non-Preconditioned Conjugate Gradient on Cell and FPGA Based Hybrid Supercomputer Nodes , 2009, 2009 17th IEEE Symposium on Field Programmable Custom Computing Machines.

[19]  David Gregg,et al.  High Performance Scientific Computing Using FPGAs with IEEE Floating Point and Logarithmic Arithmetic for Lattice QCD , 2006, 2006 International Conference on Field Programmable Logic and Applications.

[20]  Wei Zhang,et al.  Portable and scalable FPGA-based acceleration of a direct linear system solver , 2008, 2008 International Conference on Field-Programmable Technology.

[21]  Kang C. Jea,et al.  Generalized conjugate-gradient acceleration of nonsymmetrizable iterative methods , 1980 .

[22]  Vincent Heuveline,et al.  Employing a High-Level Language for Porting Numerical Applications to Reconfigurable Hardware , 2011 .

[23]  Yan Zhang,et al.  FPGA vs. GPU for sparse matrix vector multiply , 2009, 2009 International Conference on Field-Programmable Technology.

[24]  Gene H. Golub,et al.  Numerical solution of nonlinear elliptic partial differential equations by a generalized conjugate gradient method , 1976, Computing.

[25]  Siam Rfview,et al.  CONVERGENCE CONDITIONS FOR ASCENT METHODS , 2016 .

[26]  J. Shewchuk.  An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[27]  James Demmel,et al.  Performance Optimizations and Bounds for Sparse Matrix-Vector Multiply , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[28]  George A. Constantinides,et al.  A High Throughput FPGA-Based Floating Point Conjugate Gradient Implementation for Dense Matrices , 2010, TRETS.

[29]  Radoslaw Pytlak.  Preconditioned conjugate gradient algorithms with column scaling , 2008, 2008 47th IEEE Conference on Decision and Control.

[30]  P. Wolfe.  Convergence Conditions for Ascent Methods. II: Some Corrections , 1971 .

[31]  Rakesh Kumar,et al.  A hardware acceleration technique for gradient descent and conjugate gradient , 2011, 2011 IEEE 9th Symposium on Application Specific Processors (SASP).

[32]  Aljoscha Smolic,et al.  Evaluation and FPGA Implementation of Sparse Linear Solvers for Video Processing Applications , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  T. Manteuffel,et al.  Necessary and Sufficient Conditions for the Existence of a Conjugate Gradient Method , 1984 .

[34]  G.J.M. Smit,et al.  Implementing the conjugate gradient algorithm on multi-core systems , 2007, 2007 International Symposium on System-on-Chip.

[35]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[36]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[37]  Artyom Petrenko.  Accelerating an iterative Helmholtz solver using reconfigurable hardware , 2014 .

[38]  Robert Strzodka,et al.  Pipelined Mixed Precision Algorithms on FPGAs for Fast and Accurate PDE Solvers from Low Precision Components , 2006, 2006 14th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[39]  Torstein Habbestad.  An FPGA-based implementation of the Conjugate Gradient Method used to solve Large Dense Systems of Linear Equations , 2011 .

[40]  André DeHon,et al.  Floating-point sparse matrix-vector multiply for FPGAs , 2005, FPGA '05.

[41]  Warren J. Gross,et al.  FPGA architecture and implementation of sparse matrix-vector multiplication for the finite element method , 2008, Comput. Phys. Commun..

[42]  S. Mitter,et al.  The conjugate gradient method for optimal control problems , 1967 .

[43]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .