Assessment of supervised machine learning methods for fluid flows

We apply supervised machine learning techniques to a number of regression problems in fluid dynamics. Four machine learning architectures are examined in terms of their characteristics, accuracy, computational cost, and robustness for canonical flow problems. We consider the estimation of force coefficients and wakes from a limited number of sensors on the surface for flows over a cylinder and NACA0012 airfoil with a Gurney flap. The influence of the temporal density of the training data is also examined. Furthermore, we consider the use of convolutional neural network in the context of super-resolution analysis of two-dimensional cylinder wake, two-dimensional decaying isotropic turbulence, and three-dimensional turbulent channel flow. In the concluding remarks, we summarize on findings from a range of regression-type problems considered herein.

[1]  Steven L. Brunton,et al.  Machine Learning Control – Taming Nonlinear Dynamics and Turbulence , 2016, Fluid Mechanics and Its Applications.

[2]  Hugo F. S. Lui,et al.  Construction of reduced-order models for fluid flows using deep feedforward neural networks , 2019, Journal of Fluid Mechanics.

[3]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[4]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[5]  Yao Zhang,et al.  Application of Convolutional Neural Network to Predict Airfoil Lift Coefficient , 2017, ArXiv.

[6]  Arthur Albert,et al.  Regression and the Moore-Penrose Pseudoinverse , 2012 .

[7]  Omer San,et al.  A neural network approach for the blind deconvolution of turbulent flows , 2017, Journal of Fluid Mechanics.

[8]  Omer San,et al.  Extreme learning machine for reduced order modeling of turbulent geophysical flows. , 2018, Physical review. E.

[9]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[10]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[11]  J. Templeton.  Evaluation of machine learning algorithms for prediction of regions of high Reynolds averaged Navier Stokes uncertainty , 2015 .

[12]  Michael Y. Hu,et al.  Effect of data standardization on neural network training , 1996 .

[13]  D. Serre.  Matrices: Theory and Applications , 2002 .

[14]  Kai Fukami,et al.  Nonlinear mode decomposition with convolutional neural networks for fluid dynamics , 2019, Journal of Fluid Mechanics.

[15]  Vladik Kreinovich,et al.  Why Deep Neural Networks: A Possible Theoretical Explanation , 2018 .

[16]  A. Mazzino,et al.  Unraveling turbulence via physics-informed data-assimilation and spectral nudging , 2018 .

[17]  Mykel J. Kochenderfer,et al.  Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning , 2018, J. Comput. Phys..

[18]  Geoffrey E. Hinton,et al.  Learning representations of back-propagation errors , 1986 .

[19]  A. Mazzino,et al.  Inferring flow parameters and turbulent configuration with physics-informed data assimilation and spectral nudging , 2018, Physical Review Fluids.

[20]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[21]  Jian Yu,et al.  Flowfield Reconstruction Method Using Artificial Neural Network , 2019, AIAA Journal.

[22]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[23]  Hyungmin Kim,et al.  An Implantable Wireless Neural Interface System for Simultaneous Recording and Stimulation of Peripheral Nerve with a Single Cuff Electrode , 2017, Sensors.

[24]  Prakash Vedula,et al.  Subgrid modelling for two-dimensional turbulence using neural networks , 2018, Journal of Fluid Mechanics.

[25]  T. Colonius,et al.  A fast immersed boundary method using a nullspace approach and multi-domain far-field boundary conditions , 2008 .

[26]  Guang-Bin Huang,et al.  Extreme learning machine: a new learning scheme of feedforward neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[27]  Yuji Hattori,et al.  Searching for turbulence models by artificial neural network , 2016, 1607.01042.

[28]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[29]  K. Asai,et al.  Airfoil wake modification with Gurney flap at Low-Reynolds number , 2017, 1708.08500.

[30]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[31]  Petros Koumoutsakos,et al.  Machine Learning for Fluid Mechanics , 2019, Annual Review of Fluid Mechanics.

[32]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[33]  K. Taira,et al.  Network community-based model reduction for vortical flows. , 2018, Physical review. E.

[34]  T. Kijima,et al.  A new type of host compound consisting of α-zirconium phosphate and an animated cyclodextrin , 1986, Nature.

[35]  T. P. Miyanawala,et al.  A Novel Deep Learning Method for the Predictions of Current Forces on Bluff Bodies , 2018, Volume 2: CFD and FSI.

[36]  Petros Koumoutsakos,et al.  A theoretical prediction of friction drag reduction in turbulent flow by superhydrophobic surfaces , 2006 .

[37]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[38]  Yifan He,et al.  Single Image Super-Resolution Based on Multi-Scale Competitive Convolutional Neural Network , 2018, Sensors.

[39]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[40]  J. Nathan Kutz,et al.  Deep learning in fluid dynamics , 2017, Journal of Fluid Mechanics.

[41]  Kunihiko Taira,et al.  Revealing essential dynamics from high-dimensional fluid flow data and operators , 2019, 1903.01913.

[42]  Steven L. Brunton,et al.  Shallow Learning for Fluid Flow Reconstruction with Limited Sensors and Limited Data , 2019, ArXiv.

[43]  Koji Fukagata,et al.  Synthetic turbulent inflow generator using machine learning , 2018, Physical Review Fluids.

[44]  Karthik Duraisamy,et al.  Modal Analysis of Fluid Flows: Applications and Outlook , 2019, AIAA Journal.

[45]  Vassilios Theofilis,et al.  Modal Analysis of Fluid Flows: An Overview , 2017, 1702.01453.

[46]  N. Mingotti,et al.  Mixing and reaction in turbulent plumes: the limits of slow and instantaneous chemical kinetics , 2018, Journal of Fluid Mechanics.

[47]  Wisam K. Hussam,et al.  Linear stability and energetics of rotating radial horizontal convection , 2016, Journal of Fluid Mechanics.

[48]  Vladik Kreinovich,et al.  Arbitrary nonlinearity is sufficient to represent all functions by neural networks: A theorem , 1991, Neural Networks.

[49]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[50]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[51]  Jean Rabault,et al.  Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control , 2018, Journal of Fluid Mechanics.

[52]  Hossein Azizpour,et al.  Predictions of turbulent shear flows using deep neural networks , 2019, Physical Review Fluids.

[53]  David D. Cox,et al.  Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures , 2013, ICML.

[54]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[55]  Karthik Duraisamy,et al.  Turbulence Modeling in the Age of Data , 2018, Annual Review of Fluid Mechanics.

[56]  Tim Colonius,et al.  The immersed boundary method: A projection approach , 2007, J. Comput. Phys..

[57]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[58]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[59]  K. Taira,et al.  Super-resolution reconstruction of turbulent flows with machine learning , 2018, Journal of Fluid Mechanics.

[60]  Steven L. Brunton,et al.  Data-Driven Science and Engineering , 2019 .

[61]  Pedro M. Domingos.  A few useful things to know about machine learning , 2012, Commun. ACM.

[62]  Robert C. Holte,et al.  Decision Tree Instability and Active Learning , 2007, ECML.

[63]  Prasanna Balaprakash,et al.  Time-series learning of latent-space dynamics for reduced-order model closure , 2019, Physica D: Nonlinear Phenomena.

[64]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[65]  Steven L. Brunton,et al.  Network structure of two-dimensional decaying isotropic turbulence , 2016, Journal of Fluid Mechanics.

[66]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[67]  M. P. Brenner,et al.  Perspective on machine learning for advancing fluid mechanics , 2019, Physical Review Fluids.

[68]  V. Vapnik.  Pattern recognition using generalized portrait method , 1963 .

[69]  Vivek Bannore,et al.  Iterative-Interpolation Super-Resolution Image Reconstruction - A Computationally Efficient Technique , 2009, Studies in Computational Intelligence.

[70]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[71]  W. R. Peltier,et al.  Deep learning of mixing by two ‘atoms’ of stratified turbulence , 2018, Journal of Fluid Mechanics.