A performance analysis of ensemble averaging for high fidelity turbulence simulations at the strong scaling limit

Abstract We analyze the potential performance benefits of estimating expected quantities in large eddy simulations of turbulent flows using true ensembles rather than ergodic time averaging. Multiple realizations of the same flow are simulated in parallel, using slightly perturbed initial conditions to create unique instantaneous evolutions of the flow field. Each realization is then used to calculate statistical quantities. Provided each instance is sufficiently de-correlated, this approach potentially allows considerable reduction in the time to solution beyond the strong scaling limit for a given accuracy. This paper focuses on the theory and implementation of the methodology in Nek5000, a massively parallel open-source spectral element code.

[1]  Pierre Sagaut,et al.  Zonal multi‐domain RANS/LES simulations of turbulent flows , 2002 .

[2]  Elia Merzari,et al.  Accelerating the High-Fidelity Simulation of Turbulence: Ensemble Averaging , 2016 .

[3]  T. Cebeci.  Stability and Transition , 2004 .

[4]  P. Sagaut.  BOOK REVIEW: Large Eddy Simulation for Incompressible Flows. An Introduction , 2001 .

[5]  Yvonne Jaeger,et al.  Turbulence: An Introduction for Scientists and Engineers , 2015 .

[6]  Michel Schanen,et al.  On the Strong Scaling of the Spectral Element Solver Nek5000 on Petascale Systems , 2016, EASC.

[7]  Leonhard Kleiser,et al.  High-pass filtered eddy-viscosity models for large-eddy simulations of transitional and turbulent flow , 2005 .

[8]  Shmuel Einav,et al.  A laser-Doppler velocimetry study of ensemble-averaged characteristics of the turbulent near wake of a square cylinder , 1995, Journal of Fluid Mechanics.

[9]  Wolfgang Rodi,et al.  On the Simulation of Turbulent Flow Past Bluff Bodies , 1993 .

[10]  Daniele Carati.  Perspectives for ensemble average LES , 1997 .

[11]  P. Sagaut,et al.  Large-eddy simulation of a compressible flow past a deep cavity , 2003 .

[12]  P. Fischer,et al.  Petascale algorithms for reactor hydrodynamics , 2008 .

[13]  Elia Merzari,et al.  Numerical Simulation and Proper Orthogonal Decomposition of the Flow in a Counter-Flow T-Junction , 2013 .

[14]  Robert H. Shumway,et al.  Time series analysis and its applications : with R examples , 2017 .

[15]  S. A. Ragab,et al.  A Large-Eddy Simulation of the Shear-Driven Cavity Flow Using Dynamic Modeling , 1996 .

[16]  Katherine Heisey,et al.  Scaling limits for PDE-based simulation , 2015 .

[17]  Wolfgang Rodi,et al.  The flapping shear layer formed by flow separation from the forward corner of a square cylinder , 1994, Journal of Fluid Mechanics.

[18]  Daniele Carati,et al.  Ensemble averaged dynamic modeling , 1996 .

[19]  Timothy J. Tautges,et al.  Large Eddy Simulation of Thermo-Hydraulic Mixing in a T-Junction , 2013 .

[20]  Christina Freytag,et al.  Stability And Transition In Shear Flows , 2016 .

[21]  John M. Levesque,et al.  An MPI/OpenACC implementation of a high-order electromagnetics solver with GPUDirect communication , 2016, Int. J. High Perform. Comput. Appl..

[22]  P. Fischer,et al.  High-Order Methods for Incompressible Fluid Flow , 2002 .

[23]  G. Amdhal,et al.  Validity of the single processor approach to achieving large scale computing capabilities , 1967, AFIPS '67 (Spring).

[24]  Elia Merzari,et al.  Outcomes from the DOE Workshop on Turbulent Flow Simulation at the Exascale , 2016 .

[25]  Gaston H. Gonnet,et al.  Scientific Computation , 2009 .

[26]  Elia Merzari,et al.  Accurate prediction of the wall shear stress in rod bundles with the spectral element method at high Reynolds numbers , 2014 .

[27]  Elia Merzari,et al.  Large-scale simulation of nuclear reactors: Issues and perspectives , 2015 .