A Perspective on Machine Learning Methods in Turbulence Modelling

This work presents a review of the current state of research in data-driven turbulence closure modeling. It offers a perspective on the challenges and open issues, but also on the advantages and promises of machine learning methods applied to parameter estimation, model identification, closure term reconstruction and beyond, mostly from the perspective of Large Eddy Simulation and related techniques. We stress that consistency of the training data, the model, the underlying physics and the discretization is a key issue that needs to be considered for a successful ML-augmented modeling strategy. In order to make the discussion useful for non-experts in either field, we introduce both the modeling problem in turbulence as well as the prominent ML paradigms and methods in a concise and self-consistent manner. Following, we present a survey of the current data-driven model concepts and methods, highlight important developments and put them into the context of the discussed challenges.

[1]  Murray Campbell,et al.  Deep Blue , 2002, Artif. Intell..

[2]  Chao Ma,et al.  Modeling subgrid-scale force and divergence of heat flux of compressible isotropic turbulence by artificial neural network , 2019, Physical Review Fluids.

[3]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  C. Xie,et al.  Artificial neural network mixed model for large eddy simulation of compressible isotropic turbulence , 2019, Physics of Fluids.

[6]  D. Oberle,et al.  Temporal large-eddy simulation based on direct deconvolution , 2020, Physics of Fluids.

[7]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[8]  Roger Temam,et al.  Approximation of attractors, large eddy simulations and multiscale methods , 1991, Proceedings of the Royal Society of London. Series A: Mathematical and Physical Sciences.

[9]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[10]  Koji Fukagata,et al.  Synthetic turbulent inflow generator using machine learning , 2018, Physical Review Fluids.

[11]  Hua Yang,et al.  PIV-DCNN: cascaded deep convolutional neural networks for particle image velocimetry , 2017, Experiments in Fluids.

[12]  F. Sarghini,et al.  Neural networks based subgrid scale modeling in large eddy simulations , 2003 .

[13]  Geoffrey E. Hinton,et al.  Autoencoders, Minimum Description Length and Helmholtz Free Energy , 1993, NIPS.

[14]  Gopal R. Yalla,et al.  Statistical Properties of Subgrid-Scale Turbulence Models , 2021 .

[15]  Johan Larsson,et al.  Large eddy simulation with modeled wall-stress: recent progress and future directions , 2016 .

[16]  Yi Li,et al.  A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence , 2008, 0804.1703.

[17]  Rico Sennrich,et al.  Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation , 2014 .

[18]  Bernhard Schölkopf,et al.  Prior Knowledge in Support Vector Kernels , 1997, NIPS.

[19]  Jinlong Wu,et al.  Physics-informed machine learning approach for reconstructing Reynolds stress modeling discrepancies based on DNS data , 2016, 1606.07987.

[20]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[21]  Christophe Eric Corre,et al.  Subgrid-scale scalar flux modelling based on optimal estimation theory and machine-learning procedures , 2017 .

[22]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[23]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[24]  Karthik Duraisamy,et al.  A paradigm for data-driven predictive modeling using field inversion and machine learning , 2016, J. Comput. Phys..

[25]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[26]  Anikesh Pal,et al.  Deep Learning parameterization of subgrid scales in wall-bounded turbulent flows , 2019, 1905.12765.

[27]  Andrea Beck,et al.  A machine learning framework for LES closure terms , 2020, ArXiv.

[28]  G. He,et al.  Subgrid-scale model for large-eddy simulation of isotropic turbulent flows using an artificial neural network , 2019, Computers & Fluids.

[29]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[30]  J. Nathan Kutz,et al.  Deep learning in fluid dynamics , 2017, Journal of Fluid Mechanics.

[31]  R. Moser,et al.  Optimal LES formulations for isotropic turbulence , 1999, Journal of Fluid Mechanics.

[32]  T. Tatsumi Theory of Homogeneous Turbulence , 1980 .

[33]  Been Kim,et al.  Towards A Rigorous Science of Interpretable Machine Learning , 2017, 1702.08608.

[34]  Quanshi Zhang,et al.  Interpretable Convolutional Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[35]  梶島 岳夫 乱流の数値シミュレーション = Numerical simulation of turbulent flows , 2003 .

[36]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[37]  G. Santin,et al.  Kernel Methods for Surrogate Modeling , 2019, System- and Data-Driven Methods and Algorithms.

[38]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[39]  W. E,et al.  Modeling subgrid-scale forces by spatial artificial neural networks in large eddy simulation of turbulence , 2020 .

[40]  C. G. Speziale Galilean invariance of subgrid-scale stress models in the large-eddy simulation of turbulence , 1985, Journal of Fluid Mechanics.

[41]  Omer San,et al.  Spatiotemporally dynamic implicit large eddy simulation using machine learning classifiers , 2020 .

[42]  Gianluca Geraci,et al.  Optimal estimator and artificial neural network as efficient tools for the subgrid-scale scalar flux modeling , 2014 .

[43]  A. Mohan,et al.  Compressed Convolutional LSTM: An Efficient Deep Learning framework to Model High Fidelity 3D Turbulence , 2019, 1903.00033.

[44]  M. Glauser,et al.  Whither turbulence and big data in the 21st century , 2017 .

[45]  Paola Cinnella,et al.  Discovery of Algebraic Reynolds-Stress Models Using Sparse Symbolic Regression , 2019 .

[46]  Jan S. Hesthaven,et al.  An artificial neural network as a troubled-cell indicator , 2018, J. Comput. Phys..

[47]  J. Templeton Evaluation of machine learning algorithms for prediction of regions of high Reynolds averaged Navier Stokes uncertainty , 2015 .

[48]  Andrea D. Beck,et al.  A Neural Network based Shock Detection and Localization Approach for Discontinuous Galerkin Methods , 2020, J. Comput. Phys..

[49]  Yang Song,et al.  Improving the Robustness of Deep Neural Networks via Stability Training , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Karthik Duraisamy,et al.  Turbulence Modeling in the Age of Data , 2018, Annual Review of Fluid Mechanics.

[51]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[52]  Yuji Hattori,et al.  Searching for turbulence models by artificial neural network , 2016, 1607.01042.

[53]  Petros Koumoutsakos,et al.  Machine Learning for Fluid Mechanics , 2019, Annual Review of Fluid Mechanics.

[54]  Liwei Wang,et al.  The Expressive Power of Neural Networks: A View from the Width , 2017, NIPS.

[55]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[56]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[57]  Hava T. Siegelmann,et al.  Support Vector Clustering , 2002, J. Mach. Learn. Res..

[58]  Yan Wang,et al.  Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning. , 2020 .

[59]  K. Taira,et al.  Super-resolution reconstruction of turbulent flows with machine learning , 2018, Journal of Fluid Mechanics.

[60]  Brendan D. Tracey,et al.  Application of supervised learning to quantify uncertainties in turbulence and combustion modeling , 2013 .

[61]  C. D. Pruett,et al.  Toward the De-Mystification of LES , 2001 .

[62]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[63]  Omer San,et al.  A neural network approach for the blind deconvolution of turbulent flows , 2017, Journal of Fluid Mechanics.

[64]  George Em Karniadakis,et al.  NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations , 2020, J. Comput. Phys..

[65]  M. Oberlack Invariant Modelling in Large-Eddy Simulation of Turbulence , 1997 .

[66]  A. Kolmogorov A refinement of previous hypotheses concerning the local structure of turbulence in a viscous incompressible fluid at high Reynolds number , 1962, Journal of Fluid Mechanics.

[67]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[68]  Raja Giryes,et al.  Autoencoders , 2020, ArXiv.

[69]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[70]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[71]  Katepalli R. Sreenivasan,et al.  A perspective on machine learning in turbulent flows , 2020 .

[72]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[73]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[74]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[75]  John Haugeland,et al.  Artificial intelligence - the very idea , 1987 .