Weighted Block-Asynchronous Iteration on GPU-Accelerated Systems

In this paper, we analyze the potential of using weights for block-asynchronous relaxation methods on GPUs. For this purpose, we introduce different weighting techniques similar to those applied in block-smoothers for multigrid methods. For test matrices taken from the University of Florida Matrix Collection we report the convergence behavior and the total runtime for the different techniques. Analyzing the results, we observe that using weights may accelerate the convergence rate of block-asynchronous iteration considerably. While component-wise relaxation methods are seldom directly applied to systems of linear equations, using them as smoother in a multigrid framework they often provide an important contribution to finite element solvers. Since the parallelization potential of the classical smoothers like SOR and Gauss-Seidel is usually very limited, replacing them by weighted block-asynchronous smoothers may be beneficial to the overall multigrid performance. Due to the increase of heterogeneity in today's architecture designs, the significance and the need for highly parallel asynchronous smoothers is expected to grow.

[1]  Michel Dubois,et al.  Sufficient conditions for the convergence of asynchronous iterations , 1989, Parallel Computing.

[2]  Daniel B. Szyld,et al.  Asynchronous Iterations , 2011, Encyclopedia of Parallel Computing.

[3]  Dimitri P. Bertsekas,et al.  Distributed Asynchronous Relaxation Methods for Linear Network Flow Problems , 1987 .

[4]  J. Strikwerda.  A convergence theorem for chaotic asynchronous relaxation , 1997 .

[5]  D. Bertsekas.  Distributed relaxation methods for linear network flow problems , 1986, 1986 25th IEEE Conference on Decision and Control.

[6]  Jack J. Dongarra,et al.  A Block-Asynchronous Relaxation Method for Graphics Processing Units , 2011, 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & PhD Forum.

[7]  Daniel B. Szyld,et al.  Block and asynchronous two-stage methods for mildly nonlinear systems , 1999, Numerische Mathematik.

[8]  Roberto Bagnara,et al.  A Unified Proof for the Convergence of Jacobi and Gauss-Seidel Methods , 1995, SIAM Rev..

[9]  Todd Gamblin,et al.  Scaling Algebraic Multigrid Solvers: On the Road to Exascale , 2010, CHPC.

[10]  Jacques M. Bahi,et al.  Asynchronous multisplitting methods for nonlinear fixed point problems , 1997, Numerical Algorithms.

[11]  Yuan Shi,et al.  Timing Models and Local Stopping Criteria for Asynchronous Iterative Algorithms , 1999, J. Parallel Distributed Comput..

[12]  Jack J. Dongarra,et al.  Block-asynchronous Multigrid Smoothers for GPU-accelerated Systems , 2011, ICCS.

[13]  Michel Dubois,et al.  Generalized Asynchronous Iterations , 1986, CONPAR.

[14]  Michel Dubois,et al.  The Run-Time Efficiency of Parallel Asynchronous Algorithms , 1991, IEEE Trans. Computers.

[15]  Robert D. Falgout,et al.  Multigrid Smoothers for Ultra-Parallel Computing , 2011 .

[16]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[17]  J. Gillis,et al.  Matrix Iterative Analysis , 1961 .

[18]  Robert D. Falgout,et al.  Multigrid Smoothers for Ultraparallel Computing , 2011, SIAM J. Sci. Comput..