An MPI/GPU parallelization of an interior penalty discontinuous Galerkin time domain method for Maxwell's equations

[1] In this paper we discuss our approach to the MPI/GPU implementation of an Interior Penalty Discontinuous Galerkin Time domain (IPDGTD) method to solve the time dependent Maxwell's equations. In our approach, we exploit the inherent DGTD parallelism and describe a combined MPI/GPU and local time stepping implementation. This combination is aimed at increasing efficiency and reducing computational time, especially for multiscale applications. The CUDA programming model was used, together with non-blocking MPI calls to overlap communications across the network. A 10× speedup compared to CPU clusters is observed for double precision arithmetic. Finally, for p = 1 basis functions, a good scalability with parallelization efficiency of 85% for up to 40 GPUs and 80% for up to 160 CPU cores was achieved on the Ohio Supercomputer Center's Glenn cluster.

[1]  Markus Clemens,et al.  GPU Accelerated Adams–Bashforth Multirate Discontinuous Galerkin FEM Simulation of High-Frequency Electromagnetic Fields , 2010, IEEE Transactions on Magnetics.

[2]  Markus Clemens,et al.  Scalability of Higher-Order Discontinuous Galerkin FEM Computations for Solving Electromagnetic Wave Propagation Problems on GPU Clusters , 2010, IEEE Transactions on Magnetics.

[3]  J. Hesthaven,et al.  Nodal high-order methods on unstructured grids , 2002 .

[4]  David R. Smith,et al.  Metamaterial Electromagnetic Cloak at Microwave Frequencies , 2006, Science.

[5]  Jin-Fa Lee,et al.  Interior Penalty Discontinuous Galerkin Finite Element Method for the Time-Dependent First Order Maxwell's Equations , 2010, IEEE Transactions on Antennas and Propagation.

[6]  D. Arnold.  An Interior Penalty Finite Element Method with Discontinuous Elements , 1982 .

[7]  L. Fezoui,et al.  Convergence and stability of a discontinuous galerkin time-domain method for the 3D heterogeneous maxwell equations on unstructured meshes , 2005 .

[8]  George Karypis,et al.  Multilevel k-way Partitioning Scheme for Irregular Graphs , 1998, J. Parallel Distributed Comput..

[9]  Timothy C. Warburton,et al.  Nodal discontinuous Galerkin methods on graphics processors , 2009, J. Comput. Phys..

[10]  Jin-Fa Lee,et al.  Interconnect and lumped elements modeling in interior penalty discontinuous Galerkin time-domain methods , 2010, J. Comput. Phys..

[11]  E. Montseny,et al.  Dissipative terms and local time-stepping improvements in a spatial high order Discontinuous Galerkin scheme for the time-domain Maxwell's equations , 2008, J. Comput. Phys..