Improvement of the computational performance of a parallel unstructured WENO finite volume CFD code for Implicit Large Eddy Simulation

Abstract In this paper the assessment and the enhancement of the computational performance of a high-order finite volume CFD code is presented. Weighted Essentially Non-Oscillatory (WENO) schemes are considered to be from the most computationally expensive numerical frameworks, in the context of high-resolution schemes particularly on hybrid unstructured grids. The focus of this study is to assess the computational bottlenecks of the solver for the WENO schemes for Implicit Large Eddy Simulation (ILES) and optimise the performance and efficiency through a series of code modifications e.g. formula rewriting, reduction of number operations, inclusion of linear systems libraries, non-blocking communications amongst others. The code is assessed on five different HPC systems; significant speed-up is achieved ranging from 1.5 to 8.5, with very high-order schemes benefiting the most. Good scalability is also obtained up to 104 number of cores, demonstrating viability and affordability of WENO type schemes for scale resolving simulations.

[1]  João Luiz F. Azevedo,et al.  High‐order ENO and WENO schemes for unstructured grids , 2007 .

[2]  M. de la Llave Plata,et al.  Development of a multiscale LES model in the context of a modal discontinuous Galerkin method , 2016 .

[3]  Antonis F. Antoniadis,et al.  Numerical Accuracy in RANS Computations of High-Lift Multi-element Airfoil and Aicraft Configurations , 2015 .

[4]  Michael Dumbser,et al.  Quadrature-free non-oscillatory finite volume schemes on unstructured meshes for nonlinear hyperbolic systems , 2007, J. Comput. Phys..

[5]  Z. Wang,et al.  Implicit Large Eddy Simulation of Low Reynolds Number Transitional Flow over a Wing Using High-Order Spectral Difference Method , 2010 .

[6]  H. T. Huynh,et al.  A Flux Reconstruction Approach to High-Order Schemes Including Discontinuous Galerkin Methods , 2007 .

[7]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[8]  Zhi J. Wang,et al.  Spectral (Finite) Volume Method for Conservation Laws on Unstructured Grids. Basic Formulation , 2002 .

[9]  D. Drikakis,et al.  Comparison of structured- and unstructured-grid, compressible and incompressible methods using the vortex pairing problem , 2015 .

[10]  Panagiotis Tsoutsanis,et al.  High-Order Methods for Hypersonic Shock Wave Turbulent Boundary Layer Interaction Flow , 2015 .

[11]  Antony Jameson,et al.  A New Class of High-Order Energy Stable Flux Reconstruction Schemes , 2011, J. Sci. Comput..

[12]  C. Ollivier-Gooch,et al.  A high-order-accurate unstructured mesh finite-volume scheme for the advection-diffusion equation , 2002 .

[13]  Carl Ollivier-Gooch,et al.  Accuracy analysis of unstructured finite volume discretization schemes for diffusive fluxes , 2014 .

[14]  Miguel R. Visbal,et al.  Numerical Investigation of Plasma-Based Control for Low-Reynolds-Number Airfoil Flows , 2011 .

[15]  Donghyun You,et al.  A conservative finite volume method for incompressible Navier-Stokes equations on locally refined nested Cartesian grids , 2016, J. Comput. Phys..

[16]  Antony Jameson,et al.  On the Non-linear Stability of Flux Reconstruction Schemes , 2012, J. Sci. Comput..

[17]  Claus-Dieter Munz,et al.  High‐order discontinuous Galerkin spectral element methods for transitional and turbulent flow simulations , 2014 .

[18]  Chi-Wang Shu,et al.  Runge–Kutta Discontinuous Galerkin Methods for Convection-Dominated Problems , 2001, J. Sci. Comput..

[19]  Jonathan R. Bull,et al.  Simulation of the Taylor–Green Vortex Using High-Order Flux Reconstruction Schemes , 2015 .

[20]  Michael Dumbser,et al.  A unified framework for the construction of one-step finite volume and discontinuous Galerkin schemes on unstructured meshes , 2008, J. Comput. Phys..

[21]  Per-Olof Persson,et al.  Implicit Large Eddy Simulation of transition to turbulence at low Reynolds numbers using a Discontinuous Galerkin method , 2011 .

[22]  Michael Dumbser,et al.  Runge-Kutta Discontinuous Galerkin Method Using WENO Limiters , 2005, SIAM J. Sci. Comput..

[23]  Panagiotis Tsoutsanis,et al.  Azure: an advanced CFD software suite based on high-resolution and high-order methods , 2015 .

[24]  Christer Fureby,et al.  Simulation of transition and turbulence decay in the Taylor–Green vortex , 2007 .

[25]  Miguel R. Visbal,et al.  Comparative study of implicit and subgrid‐scale model large‐eddy simulation techniques for low‐Reynolds number airfoil applications , 2013 .

[26]  W. A. Li,et al.  Quadrature‐Free Non‐Oscillation Finite Volume Scheme for Solving Navier‐Stokes Equations on Unstructured grids , 2011 .

[27]  Z. Wang,et al.  Spectral (Finite) Volume Method for Conservation Laws on Unstructured Grids , 2002 .

[28]  Fermín Navarrina,et al.  A new shock-capturing technique based on Moving Least Squares for higher-order numerical schemes on unstructured grids , 2010 .

[29]  Panagiotis Tsoutsanis,et al.  Assessment of high-order finite volume methods on unstructured meshes for RANS solutions of aeronautical configurations , 2017 .

[30]  Andreas Haselbacher,et al.  A WENO Reconstruction Algorithim for Unstructured Grids Based on Explicit Stencil Construction , 2005 .

[31]  Sigal Gottlieb,et al.  On High Order Strong Stability Preserving Runge–Kutta and Multi Step Time Discretizations , 2005, J. Sci. Comput..

[32]  Zhiliang Xu,et al.  Hierarchical reconstruction for discontinuous Galerkin methods on unstructured grids with a WENO-type linear reconstruction and partial neighboring cells , 2009, J. Comput. Phys..

[33]  C. Ollivier-Gooch.  Quasi-ENO Schemes for Unstructured Meshes Based on Unlimited Data-Dependent Least-Squares Reconstruction , 1997 .

[34]  Michael Dumbser,et al.  Central Weighted ENO Schemes for Hyperbolic Conservation Laws on Fixed and Moving Unstructured Meshes , 2017, SIAM J. Sci. Comput..

[35]  Freddie D. Witherden,et al.  PyFR: An open source framework for solving advection-diffusion type problems on streaming architectures using the flux reconstruction approach , 2013, Comput. Phys. Commun..

[36]  Clinton P. T. Groth,et al.  High-Order Solution-Adaptive Central Essentially Non-Oscillatory (CENO) Method for Viscous Flows , 2011 .

[37]  Vladimir A. Titarev,et al.  WENO schemes on arbitrary mixed-element unstructured meshes in three space dimensions , 2011, J. Comput. Phys..

[38]  Dimitris Drikakis,et al.  WENO schemes on arbitrary unstructured meshes for laminar, transitional and turbulent flows , 2014, J. Comput. Phys..

[39]  Alessandro Colombo,et al.  Linearly implicit Rosenbrock-type Runge–Kutta schemes applied to the Discontinuous Galerkin solution of compressible and incompressible unsteady flows , 2015 .

[40]  Michael Dumbser,et al.  High order ADER schemes for a unified first order hyperbolic formulation of continuum mechanics: Viscous heat-conducting fluids and elastic solids , 2015, J. Comput. Phys..

[41]  G. Russo,et al.  Central WENO schemes for hyperbolic conservation laws on fixed and moving unstructured meshes , 2016 .

[42]  Dimitris Drikakis,et al.  WENO schemes for mixed-elementunstructured meshes , 2010 .

[43]  Brian C. Vermeire,et al.  On the behaviour of fully-discrete flux reconstruction schemes , 2017 .

[44]  Zhi J. Wang,et al.  Spectral (finite) volume method for conservation laws on unstructured grids IV: extension to two-dimensional systems , 2004 .

[45]  Zhi J. Wang,et al.  Implicit Large Eddy Simulation of Transitional Flow over a SD7003 Wing Using High-order Spectral Difference Method , 2010 .

[46]  Hiroaki Nishikawa.  Robust and accurate viscous discretization via upwind scheme – I: Basic principle , 2011 .

[47]  João Luiz F. Azevedo,et al.  Improved High-Order Spectral Finite Volume Method Implementation for Aerodynamic Flows , 2009 .

[48]  Brian C. Vermeire,et al.  On the properties of energy stable flux reconstruction schemes for implicit large eddy simulation , 2016, J. Comput. Phys..

[49]  Leland Jameson,et al.  Numerical Convergence Study of Nearly Incompressible, Inviscid Taylor–Green Vortex Flow , 2005, J. Sci. Comput..

[50]  John M. Dennis,et al.  Towards an efficient and scalable discontinuous Galerkin atmospheric model , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[51]  E. Toro.  Riemann Solvers and Numerical Methods for Fluid Dynamics , 1997 .

[52]  Claus-Dieter Munz,et al.  A contribution to the construction of diffusion fluxes for finite volume and discontinuous Galerkin schemes , 2007, J. Comput. Phys..

[53]  Miguel R. Visbal,et al.  High-Fidelity Simulation of Transitional Flows Past a Plunging Airfoil , 2009 .

[54]  Zhiliang Xu,et al.  Hierarchical reconstruction for spectral volume method on unstructured grids , 2009, J. Comput. Phys..