Neural Ideal Large Eddy Simulation: Modeling Turbulence with Neural Stochastic Differential Equations

We introduce a data-driven learning framework that assimilates two powerful ideas: ideal large eddy simulation (LES) from turbulence closure modeling and neural stochastic differential equations (SDE) for stochastic modeling. The ideal LES models the LES flow by treating each full-order trajectory as a random realization of the underlying dynamics, as such, the effect of small-scales is marginalized to obtain the deterministic evolution of the LES state. However, ideal LES is analytically intractable. In our work, we use a latent neural SDE to model the evolution of the stochastic process and an encoder-decoder pair for transforming between the latent space and the desired ideal flow field. This stands in sharp contrast to other types of neural parameterization of closure models where each trajectory is treated as a deterministic realization of the dynamics. We show the effectiveness of our approach (niLES - neural ideal LES) on a challenging chaotic dynamical system: Kolmogorov flow at a Reynolds number of 20,000. Compared to competing methods, our method can handle non-uniform geometries using unstructured meshes seamlessly. In particular, niLES leads to trajectories with more accurate statistics and enhances stability, particularly for long-horizon rollouts.

[1]  Suman V. Ravuri,et al.  GraphCast: Learning skillful medium-range global weather forecasting , 2022, ArXiv.

[2]  L. Zanna,et al.  Benchmarking of Machine Learning Ocean Subgrid Parameterizations in an Idealized Model , 2022, Journal of Advances in Modeling Earth Systems.

[3]  Lingxi Xie,et al.  Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast , 2022, ArXiv.

[4]  Jamie A. Smith,et al.  Learning to correct spectral methods for simulating turbulent flows , 2022, ArXiv.

[5]  D. You,et al.  Neural-network-based mixed subgrid-scale model for turbulent flow , 2022, Journal of Fluid Mechanics.

[6]  W. Bhimji,et al.  Long-term stability and generalization of observationally-constrained stochastic data-driven models for geophysical turbulence , 2022, Environmental Data Science.

[7]  K. Azizzadenesheli,et al.  FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators , 2022, ArXiv.

[8]  Li-Wei Chen,et al.  Learned turbulence modelling with differentiable fluid solvers: physics-based loss functions and optimisation horizons , 2022, Journal of Fluid Mechanics.

[9]  Daniel E. Worrall,et al.  Message Passing Neural PDE Solvers , 2022, ICLR.

[10]  Patrick Kidger On Neural Differential Equations , 2022, ArXiv.

[11]  A. Chattopadhyay,et al.  Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES , 2022, Physica D: Nonlinear Phenomena.

[12]  T. Pfaff,et al.  Learned Coarse Models for Efficient Turbulence Simulation , 2021, ArXiv.

[13]  J. Malik,et al.  MViTv2: Improved Multiscale Vision Transformers for Classification and Detection , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Patrick Kidger,et al.  Efficient and Accurate Gradients for Neural SDEs , 2021, NeurIPS.

[15]  Justin A. Sirignano,et al.  Embedded training of neural-network sub-grid-scale turbulence models , 2021, Physical Review Fluids.

[16]  Christoph Feichtenhofer,et al.  Multiscale Vision Transformers , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  Haecheon Choi,et al.  Toward neural-network-based large eddy simulation: application to turbulent channel flow , 2021, Journal of Fluid Mechanics.

[18]  P. Koumoutsakos,et al.  Computing foaming flows across scales: From breaking waves to microfluidics , 2021, Science advances.

[19]  A. Chattopadhyay,et al.  Stable a posteriori LES of 2D turbulence using convolutional neural networks: Backscattering analysis and generalization to higher Re via transfer learning , 2021, J. Comput. Phys..

[20]  Ricky T. Q. Chen,et al.  Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equations , 2021, AISTATS.

[21]  Patrick Kidger,et al.  Neural SDEs as Infinite-Dimensional GANs , 2021, ICML.

[22]  Stephan Hoyer,et al.  Machine learning–accelerated computational fluid dynamics , 2021, Proceedings of the National Academy of Sciences.

[23]  Yifei Guan,et al.  Data-driven subgrid-scale modeling of forced Burgers turbulence using deep learning with generalization to higher Reynolds numbers via transfer learning , 2020, Physics of Fluids.

[24]  A. Beck,et al.  A perspective on machine learning methods in turbulence modeling , 2020, GAMM-Mitteilungen.

[25]  Nikola B. Kovachki,et al.  Fourier Neural Operator for Parametric Partial Differential Equations , 2020, ICLR.

[26]  Etienne Mémin,et al.  Stochastic Modelling of Turbulent Flows for Numerical Simulations , 2020, Fluids.

[27]  Nils Thuerey,et al.  Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers , 2020, NeurIPS.

[28]  Kai Fukami,et al.  Probabilistic neural networks for fluid flow surrogate modeling and data recovery , 2020 .

[29]  Kai Fukami,et al.  Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows , 2020, Journal of Fluid Mechanics.

[30]  S. Girimaji,et al.  Turbulence closure modeling with data-driven techniques: physical compatibility and consistency considerations , 2020, New Journal of Physics.

[31]  Jure Leskovec,et al.  Learning to Simulate Complex Physics with Graph Networks , 2020, ICML.

[32]  Rui Wang,et al.  Towards Physics-informed Deep Learning for Turbulent Flow Prediction , 2019, KDD.

[33]  Stephan Hoyer,et al.  Learning data-driven discretizations for partial differential equations , 2018, Proceedings of the National Academy of Sciences.

[34]  Maxim Raginsky,et al.  Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit , 2019, ArXiv.

[35]  Jamey D. Jacob,et al.  Sub-grid scale model classification and blending through deep learning , 2018, Journal of Fluid Mechanics.

[36]  K. Taira,et al.  Super-resolution reconstruction of turbulent flows with machine learning , 2018, Journal of Fluid Mechanics.

[37]  Siddhartha Mishra,et al.  A machine learning framework for data driven acceleration of computations of differential equations , 2018, ArXiv.

[38]  Yoshua Bengio,et al.  On the Spectral Bias of Neural Networks , 2018, ICML.

[39]  Karthik Duraisamy,et al.  Turbulence Modeling in the Age of Data , 2018, Annual Review of Fluid Mechanics.

[40]  Karthik Duraisamy,et al.  Data-Driven Discovery of Closure Models , 2018, SIAM J. Appl. Dyn. Syst..

[41]  P. Durbin Some Recent Developments in Turbulence Closure Modeling , 2018 .

[42]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[43]  Emmanuel Leriche,et al.  UDNS or LES, That Is the Question , 2015 .

[44]  Peter Bauer,et al.  The quiet revolution of numerical weather prediction , 2015, Nature.

[45]  R. Courant,et al.  On the Partial Difference Equations, of Mathematical Physics , 2015 .

[46]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[47]  Andrew J. Majda,et al.  Stochastic superparameterization in quasigeostrophic turbulence , 2013, J. Comput. Phys..

[48]  Andrew J Majda,et al.  Efficient stochastic superparameterization for geophysical turbulence , 2013, Proceedings of the National Academy of Sciences.

[49]  Eric Vanden-Eijnden,et al.  Subgrid-Scale Parameterization with Conditional Markov Chains , 2008 .

[50]  Yi Li,et al.  A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence , 2008, 0804.1703.

[51]  Gregory A Voth,et al.  Multiscale modeling of biomolecular systems: in serial and in parallel. , 2007, Current opinion in structural biology.

[52]  Peter Lynch,et al.  The Emergence of Numerical Weather Prediction: Richardson's Dream , 2006 .

[53]  S. Pope Ten questions concerning the large-eddy simulation of turbulent flows , 2004 .

[54]  Hugh Maurice Blackburn,et al.  Spectral element filtering techniques for large eddy simulation with dynamic estimation , 2003 .

[55]  P. Fischer,et al.  High-Order Methods for Incompressible Fluid Flow , 2002 .

[56]  Thomas J. R. Hughes,et al.  Large eddy simulation of turbulent channel flows by the variational multiscale method , 2001 .

[57]  Julia S. Mullen,et al.  Filter-based stabilization of spectral element methods , 2001 .

[58]  R. Moser,et al.  Optimal LES formulations for isotropic turbulence , 1999, Journal of Fluid Mechanics.

[59]  T. Hughes,et al.  The variational multiscale method—a paradigm for computational mechanics , 1998 .

[60]  Peter Bradshaw,et al.  Turbulence modeling with application to turbomachinery , 1996 .

[61]  G. Berkooz An observation on probability density equations, or, when do simulations reproduce statistics? , 1994 .

[62]  P. Moin,et al.  A dynamic subgrid‐scale eddy viscosity model , 1990 .

[63]  C. E. Leith,et al.  Stochastic backscatter in a subgrid-scale model: Plane shear mixing layer , 1990 .

[64]  Yassin A. Hassan,et al.  Approximation of turbulent conditional averages by stochastic estimation , 1989 .

[65]  Nadine Aubry,et al.  The dynamics of coherent structures in the wall region of a turbulent boundary layer , 1988, Journal of Fluid Mechanics.

[66]  S. Orszag Analytical theories of turbulence , 1970, Journal of Fluid Mechanics.

[67]  L. Zanna,et al.  Stochastic Deep Learning parameterization of Ocean Momentum Forcing , 2021 .

[68]  Ricky T. Q. Chen,et al.  Scalable Gradients and Variational Inference for Stochastic Differential Equations , 2019, AABI.

[69]  A. P. Siebesma,et al.  Climate goals and computing the future of clouds , 2017 .

[70]  Stanislav Boldyrev,et al.  Two-dimensional turbulence , 1980 .

[71]  Pierre Sagaut Large Eddy Simulation for Incompressible Flows. An Introduction , 2001 .

[72]  T. Hughes,et al.  Large Eddy Simulation and the variational multiscale method , 2000 .

[73]  Javier Jiménez,et al.  Large-Eddy Simulations: Where Are We and What Can We Expect? , 2000 .

[74]  R. Adrian Stochastic Estimation of the Structure of Turbulent Fields , 1996 .

[75]  A. Kolmogorov,et al.  The local structure of turbulence in incompressible viscous fluid for very large Reynolds numbers , 1991, Proceedings of the Royal Society of London. Series A: Mathematical and Physical Sciences.

[76]  S. Pope PDF methods for turbulent reactive flows , 1985 .

[77]  C.E. Shannon,et al.  Communication in the Presence of Noise , 1949, Proceedings of the IRE.

[78]  Ronald Adrian,et al.  On the role of conditional averages in turbulence theory. , 1975 .

[79]  Caskey,et al.  GENERAL CIRCULATION EXPERIMENTS WITH THE PRIMITIVE EQUATIONS I . THE BASIC EXPERIMENT , 1962 .