Acceleration of the Dual-Field Domain Decomposition Algorithm Using MPI–CUDA on Large-Scale Computing Systems

It is well known that graphics processing units (GPUs) are able to accelerate highly parallelizable algorithms with a high speedup. However, for less-parallelizable algorithms such as the finite element method, novel schemes are needed to achieve a high speedup. In this paper, the dual-field domain decomposition (DFDD) method based on element-level decomposition (DFDD-ELD) is accelerated on a large GPU cluster. By using element-level subdomains, the DFDD-ELD computation can be easily mapped onto GPU's granular processors and is thus highly parallelizable. Various electromagnetic problems are simulated to demonstrate the speedup and scalability of DFDD-ELD on a GPU cluster. With a careful GPU memory arrangement and thread allocation, we are able to achieve a significant speedup by utilizing GPUs in a message-passing interface (MPI)-based cluster environment. The same acceleration strategy can be applied to the acceleration of the discontinuous Galerkin time-domain (DGTD) algorithms.

[1]  Jian-Ming Jin,et al.  A novel dual-field time-domain finite-element domain-decomposition method for computational electromagnetics , 2006 .

[2]  Jian-Ming Jin,et al.  An OpenMP-CUDA Implementation of Multilevel Fast Multipole Algorithm for Electromagnetic Simulation on Multi-GPU Computing Systems , 2013, IEEE Transactions on Antennas and Propagation.

[3]  Atef Z. Elsherbeni,et al.  Compute unified device architecture (CUDA) based finite-difference time-domain (FDTD) implementation , 2010 .

[4]  Danilo De Donno,et al.  MPIE/MoM Acceleration With a General-Purpose Graphics Processing Unit , 2012, IEEE Transactions on Microwave Theory and Techniques.

[5]  Jian-Ming Jin,et al.  A New Dual-Primal Domain Decomposition Approach for Finite Element Simulation of 3-D Large-Scale Electromagnetic Problems , 2007, IEEE Transactions on Antennas and Propagation.

[6]  Shaojing Li,et al.  Fast evaluation of Helmholtz potential on graphics processing units (GPUs) , 2010, J. Comput. Phys..

[7]  Kan Xu,et al.  GPU Accelerated Unconditionally Stable Crank-Nicolson FDTD Method for the Analysis of Three-Dimensional Microwave Circuits , 2010 .

[8]  Jian-Ming Jin,et al.  A Comparative Study of Three Finite Element-Based Explicit Numerical Schemes for Solving Maxwell's Equations , 2011, IEEE Transactions on Antennas and Propagation.

[9]  Jian-Ming Jin,et al.  Modeling of doubly lossy and dispersive media with the time‐domain finite‐element dual‐field domain‐decomposition algorithm , 2013 .

[10]  Danilo De Donno,et al.  Parallel efficient method of moments exploiting graphics processing units , 2010 .

[11]  Z. Nie,et al.  Acceleration of the Method of Moments Calculations by Using Graphics Processing Units , 2008, IEEE Transactions on Antennas and Propagation.

[12]  C. Potratz,et al.  Time-Domain Field and Scattering Parameter Computation in Waveguide Structures by GPU-Accelerated Discontinuous-Galerkin Method , 2011, IEEE Transactions on Microwave Theory and Techniques.

[13]  T Topa,et al.  Using GPU With CUDA to Accelerate MoM-Based Electromagnetic Simulation of Wire-Grid Models , 2011, IEEE Antennas and Wireless Propagation Letters.

[14]  Jin-Fa Lee,et al.  Discontinuous Galerkin Time Domain for Maxwell's equations on GPUs , 2010, 2010 URSI International Symposium on Electromagnetic Theory.

[15]  Jian-Ming Jin,et al.  Parallel implementation of the FETI-DPEM algorithm for general 3D EM simulations , 2009, J. Comput. Phys..

[16]  Shaojing Li,et al.  Fast Electromagnetic Integral-Equation Solvers on Graphics Processing Units , 2012, IEEE Antennas and Propagation Magazine.

[17]  S Schomann,et al.  Local Timestepping Techniques Using Taylor Expansion for Modeling Electromagnetic Wave Propagation With Discontinuous Galerkin-FEM , 2010, IEEE Transactions on Magnetics.

[18]  Rick Siow Mong Goh,et al.  Implementation of the FDTD Method Based on Lorentz-Drude Dispersive Model on GPU for Plasmonics Applications , 2011 .

[19]  Ming-Feng Xue,et al.  Nonconformal FETI-DP methods for numerical analysis of large-scale electromagnetic problems , 2012, 2012 International Conference on Electromagnetics in Advanced Applications.

[20]  Timothy C. Warburton,et al.  Nodal discontinuous Galerkin methods on graphics processors , 2009, J. Comput. Phys..

[21]  Bing-Hao Zeng,et al.  Modified Antipodal Fermi antenna with piecewise-linear approximation , 2010, 2010 International Conference on Applications of Electromagnetism and Student Innovation Competition Awards (AEM2C).

[22]  Markus Clemens,et al.  GPU Accelerated Adams–Bashforth Multirate Discontinuous Galerkin FEM Simulation of High-Frequency Electromagnetic Fields , 2010, IEEE Transactions on Magnetics.

[23]  Jie Cheng,et al.  Programming Massively Parallel Processors. A Hands-on Approach , 2010, Scalable Comput. Pract. Exp..

[24]  M. Mrozowski,et al.  How to Render FDTD Computations More Effective Using a Graphics Accelerator , 2009, IEEE Transactions on Magnetics.

[25]  F. Costen,et al.  Development of a CUDA Implementation of the 3D FDTD Method , 2012, IEEE Antennas and Propagation Magazine.

[26]  Markus Clemens,et al.  Scalability of Higher-Order Discontinuous Galerkin FEM Computations for Solving Electromagnetic Wave Propagation Problems on GPU Clusters , 2010, IEEE Transactions on Magnetics.

[27]  Raj Mittra,et al.  New Development of Parallel Conformal FDTD Method in Computational Electromagnetics Engineering , 2011, IEEE Antennas and Propagation Magazine.

[28]  Jian-Ming Jin,et al.  A New Explicit Time-Domain Finite-Element Method Based on Element-Level Decomposition , 2006, IEEE Transactions on Antennas and Propagation.

[29]  Markus Clemens,et al.  Accelerating Multi GPU Based Discontinuous Galerkin FEM Computations for Electromagnetic Radio Frequency Problems , 2010 .

[30]  Danilo De Donno,et al.  Introduction to GPU Computing and CUDA Programming: A Case Study on FDTD [EM Programmer's Notebook] , 2010 .

[31]  Jianming Jin,et al.  Finite Element Analysis of Antennas and Arrays , 2008 .

[32]  Jian-Ming Jin,et al.  GPU accelerated finite-element computation for electromagnetic analysis , 2014, IEEE Antennas and Propagation Magazine.

[33]  Jian-Ming Jin,et al.  A Vector Dual-Primal Finite Element Tearing and Interconnecting Method for Solving 3-D Large-Scale Electromagnetic Problems , 2006 .

[34]  Jian-Ming Jin,et al.  A dual-field domain-decomposition method for the time-domain finite-element analysis of large finite arrays , 2007, J. Comput. Phys..

[35]  Karsten Schwan,et al.  Keeneland: Bringing Heterogeneous GPU Computing to the Computational Science Community , 2011, Computing in Science & Engineering.