Direct Numerical Simulation of Incompressible Flows on Unstructured Meshes Using Hybrid CPU/GPU Supercomputers☆

Abstract This paper describes a hybrid MPI-CUDA parallelization strategy for the direct numerical simulation of incompressible flows using unstructured meshes. Our in-house MPI-based unstructured CFD code has been extended in order to increase its performance by means of GPU co-processors. Therefore, the main goal of this work is to take advantage of the current hybrid supercomputers to increase our computing capabilities. CUDA is used to perform the calculations on the GPU devices and MPI to handle the communications between them. The main drawback for the performance is the slowdown produced by the MPI communication episodes. Consequently, overlapping strategies, to hide MPI communication costs under GPU computations, are studied in detail with the aim to achieve scalability when executing the code on multiple nodes.

[1]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[2]  Oriol Lehmkuhl,et al.  Assessment of the symmetry-preserving regularization model on complex flows using unstructured grids , 2012 .

[3]  Kyriakos C. Giannakoglou,et al.  CFD-based analysis and two-level aerodynamic optimization on Graphics Processing Units , 2010 .

[4]  A. Chorin.  Numerical solution of the Navier-Stokes equations , 1968 .

[5]  Ali Khajeh-Saeed,et al.  Direct numerical simulation of turbulence using GPU accelerated supercomputers , 2013, J. Comput. Phys..

[6]  Paulius Micikevicius,et al.  3D finite difference computation on GPUs using CUDA , 2009, GPGPU-2.

[7]  F. Xavier Trias,et al.  Parallel direct Poisson solver for discretisations with one Fourier diagonalisable direction , 2011, J. Comput. Phys..

[8]  A. Veldman,et al.  Symmetry-preserving discretization of turbulent flow , 2003 .

[9]  R. Borrell,et al.  TermoFluids: A new Parallel unstructured CFD code for the simulation of turbulent industrial problems on low cost PC Cluster , 2009 .

[10]  Kyriakos C. Giannakoglou,et al.  Unsteady CFD computations using vertex‐centered finite volumes for unstructured grids on Graphics Processing Units , 2011 .

[11]  Dominik L. Michels,et al.  Sparse-Matrix-CG-Solver in CUDA , 2011 .

[12]  Mark J. Harris.  Fast fluid dynamics simulation on the GPU , 2005, SIGGRAPH Courses.

[13]  F. Xavier Trias,et al.  Parallel algorithms for Sn transport sweeps on unstructured meshes , 2013, J. Comput. Phys..