GPGPU implementation of mixed spectral-finite difference computational code for the numerical integration of the three-dimensional time-dependent incompressible Navier–Stokes equations

Abstract In the recent past, High-Performance-Computing machines and in particular Computational-Fluid-Dynamics hardware, have been dominated by architectures that mainly included clusters of processors communicating on high-speed interconnects. Since a few years though, cluster performance has reached an upper limit for many technical codes. The scalar-processor ability has saturated, and the sustained Flops often result to be only a fraction of the declared peak values. In an attempt to overcome these difficulties, the electronic industry has unveiled new trends. Since about 2004, there has been a transition taking place in high-performance scientific computing, with the advent of multicore- and manycore chips, so that the different software models for parallel programming needed to be revisited. In the present work these issues are addressed with reference to the field of fluid-flow investigation through the numerical integration of the Navier–Stokes equations. A mixed spectral-finite difference computational code for the numerical solution of the three-dimensional time-dependent incompressible Navier–Stokes equations (with no forcing), is implemented on a high-performance hybrid CPU/GPGPU computing system. The computational algorithm is thought for the execution of accurate calculations at high values of the flow Reynolds number, so allowing the investigation of turbulence with the method of the Direct Numerical Simulation (no turbulence models in the system of the governing equations). The performance of different code implementations has been measured as related to the case of the flow of a viscous incompressible fluid in a plane channel, a problem that has a long tradition in the field of wall-bounded turbulence research, so becoming a reference case for the study of turbulence via Direct Numerical Simulation. Results are presented in terms of parallel performance of the computational code on different hardware partitions of the computing system, as related to corresponding different sizes of the computational grid, showing that remarkably-high computational performance is reached, mainly in virtue of the presence of the GPGPU boards in the computing-hardware architecture.

[1]  G. Karniadakis,et al.  Recent advances in parallel spectral element simulation of unsteady incompressible flows , 1988 .

[2]  Lorenzo Muttoni,et al.  Performance evaluation of a Windows NT based PC cluster for high performance computing , 2004, J. Syst. Archit..

[3]  Paolo Cremonesi,et al.  Analysis and implementation of a parallelization strategy on a Navier-Stokes solver for shear flow simulations , 2001, Parallel Comput..

[4]  Ji Xu,et al.  Efficient 3D DNS of gas–solid flows on Fermi GPGPU , 2012 .

[5]  Leonardo Primavera,et al.  DIRECT NUMERICAL SIMULATION OF TURBULENT CHANNEL FLOW WITH MIXED SPECTRAL-FINITE DIFFERENCE TECHNIQUE , 2007 .

[6]  Inanc Senocak,et al.  CUDA Implementation of a Navier-Stokes Solver on Multi-GPU Desktop Platforms for Incompressible Flows , 2009 .

[7]  N. Kasagi,et al.  Direct Numerical Simulation of Passive Scalar Field in a Turbulent Channel Flow , 1992 .

[8]  Javier Jiménez,et al.  Scaling of the velocity fluctuations in turbulent channels up to Reτ=2003 , 2006 .

[9]  Yuichi Matsuo,et al.  Direct Numerical Simulation of a Fully Developed Turbulent Channel Flow With Respect to the Reynolds Number Dependence , 2001 .

[10]  Ryutaro Himeno,et al.  A parallel 3D unsteady incompressible flow solver on VPP700 , 2001, Parallel Comput..

[11]  Amit Basu,et al.  A Parallel Algorithm for Spectral Solution of the Three-Dimensional Navier-Stokes Equations , 1994, Parallel Comput..

[12]  Joel H. Ferziger,et al.  HYBRID SPECTRAL FINITE DIFFERENCE SIMULATIONS OF STRATIFIED TURBULENT FLOWS ON DISTRIBUTED MEMORY ARCHITECTURES , 1997 .

[13]  Michael Prestin,et al.  A parallel Navier-Stokes solver: The Meiko implementation , 1995, The Journal of Supercomputing.

[14]  R. Pelz.  The parallel Fourier pseudospectral method , 1991 .

[15]  Neil D. Sandham,et al.  Wall Pressure and Shear Stress Spectra from Direct Simulations of Channel Flow , 2006 .

[16]  Jay Hoeflinger,et al.  Producing scalable performance with OpenMP: Experiments with two CFD applications , 2001, Parallel Comput..

[17]  Dennis C. Jespersen,et al.  A Computational Fluid Dynamics Algorithm On a Massively Parallel Computer , 1989, Int. J. High Perform. Comput. Appl..

[18]  Steven A. Orszag,et al.  A case study in parallel computing: I. Homogeneous turbulence on a hypercube , 1991 .

[19]  Javier Jiménez,et al.  Spectra of the very large anisotropic scales in turbulent channels , 2003 .

[20]  George Em Karniadakis,et al.  Parallel benchmarks of turbulence in complex geometries , 1996 .

[21]  Marc Garbey,et al.  A parallel solver for unsteady incompressible 3D Navier-Stokes equations , 2001, Parallel Comput..

[22]  Jeffrey S. Reeve,et al.  Evaluation of a Spectral Element CFD Code on Parallel Architectures , 1995, Parallel Comput..

[23]  G. Alfonsi.  RESULTS OF NUMERICAL INTEGRATION OF THE NAVIER-STOKES EQUATIONS IN THE FIELD OF DIRECT SIMULATION OF TURBULENCE , 2011 .

[24]  Rod A. Fatoohi.  Adapting a Navier-Stokes solver for three parallel machines , 2005, The Journal of Supercomputing.

[25]  Mitsuo Yokokawa,et al.  Scalability of hybrid programming for a CFD code on the Earth Simulator , 2004, Parallel Comput..

[26]  John B. McLaughlin,et al.  Large‐scale computer simulation of fully developed turbulent channel flow with heat transfer , 1991 .

[27]  Marco Briscolini,et al.  A Parallel Implementation of a 3-D Pseudospectral Based Code on the IBM 9076 Scalable POWERParallel System , 1995, Parallel Comput..

[28]  Shiyi Chen,et al.  High‐resolution turbulent simulations using the Connection Machine‐2 , 1992 .

[29]  Thomas J. R. Hughes,et al.  A data parallel finite element method for computational fluid dynamics on the Connection Machine system , 1992 .

[30]  John L. Lumley,et al.  Viscous Sublayer and Adjacent Wall Region in Turbulent Pipe Flow , 1967 .

[31]  Spencer J. Sherwin,et al.  Parallel performance of the coarse space linear vertex solver and low energy basis preconditioner for spectral/hp elements , 2009, Parallel Comput..

[32]  Giuseppe Passoni,et al.  Analysis of hybrid algorithms for the Navier–Stokes equations with respect to hydrodynamic stability theory , 2002 .

[33]  Tamer M. Wasfy,et al.  Parallel finite element computation of unsteady incompressible flows , 1998 .

[34]  Graham Pullan,et al.  An Accelerated 3D Navier–Stokes Solver for Flows in Turbomachines , 2009 .

[35]  Vijay K. Naik,et al.  Parallelization of a Class of Implicit Finite Difference Schemes in Computational Fluid Dynamics , 1993, Int. J. High Speed Comput..

[36]  TURBULENT EVENTS IN A WALL-BOUNDED TURBULENT FLOW , 2012 .

[37]  Shin-Jeong Kang,et al.  SCALING OF FINE SCALE EDDIES IN TURBULENT CHANNEL FLOWS UP TO Reτ =800 , 2004, Proceeding of Third Symposium on Turbulence and Shear Flow Phenomena.

[38]  Sanjay Mittal,et al.  Parallel finite element computation of incompressible flows , 2009, Parallel Comput..

[39]  Giuseppe Passoni,et al.  A spectral–finite difference solution of the Navier–Stokes equations in three dimensions , 1998 .

[40]  Parviz Moin,et al.  An improvement of fractional step methods for the incompressible Navier-Stokes equations , 1991 .

[41]  Kyriakos C. Giannakoglou,et al.  CFD-based analysis and two-level aerodynamic optimization on Graphics Processing Units , 2010 .

[42]  Nobuhide Kasagi,et al.  Reynolds Number Effect on Wall Turbulence: Toward Effective Feedback Control , 2002 .

[43]  John Kim,et al.  DIRECT NUMERICAL SIMULATION OF TURBULENT CHANNEL FLOWS UP TO RE=590 , 1999 .

[44]  P. Moin,et al.  Turbulence statistics in fully developed channel flow at low Reynolds number , 1987, Journal of Fluid Mechanics.

[45]  R. A. Antonia,et al.  Low-Reynolds-number effects in a fully developed turbulent channel flow , 1992, Journal of Fluid Mechanics.

[46]  Suchuan Dong,et al.  Dual-level parallelism for high-order CFD methods , 2004, Parallel Comput..

[47]  Javier Jiménez,et al.  Scaling of the energy spectra of turbulent channels , 2003, Journal of Fluid Mechanics.

[48]  Charles A. Sleicher,et al.  Direct simulation of turbulent flow and heat transfer in a channel. Part I: Smooth walls , 1993 .

[49]  J. Xu,et al.  Benchmarks on tera-scalable models for DNS of turbulent channel flow , 2007, Parallel Comput..

[50]  G. Alfonsi.  On Direct Numerical Simulation of Turbulent Flows , 2011 .

[51]  Giuseppe Passoni,et al.  A wavenumber parallel computational code for the numerical integration of the Navier-Stokes equations , 1999, Parallel Comput..

[52]  Kyriakos C. Giannakoglou,et al.  Unsteady CFD computations using vertex‐centered finite volumes for unstructured grids on Graphics Processing Units , 2011 .

[53]  William Gropp,et al.  High-performance parallel implicit CFD , 2001, Parallel Comput..