Machine Learning for Fluid Mechanics

The field of fluid mechanics is rapidly advancing, driven by unprecedented volumes of data from field measurements, experiments and large-scale simulations at multiple spatiotemporal scales. Machine learning offers a wealth of techniques to extract information from data that could be translated into knowledge about the underlying fluid mechanics. Moreover, machine learning algorithms can augment domain knowledge and automate tasks related to flow control and optimization. This article presents an overview of past history, current developments, and emerging opportunities of machine learning for fluid mechanics. It outlines fundamental machine learning methodologies and discusses their uses for understanding, modeling, optimizing, and controlling fluid flows. The strengths and limitations of these methods are addressed from the perspective of scientific inquiry that considers data as an inherent part of modeling, experimentation, and simulation. Machine learning provides a powerful information processing framework that can enrich, and possibly even transform, current lines of fluid mechanics research and industrial applications.

[1]  Jinlong Wu,et al.  Physics-informed machine learning approach for reconstructing Reynolds stress modeling discrepancies based on DNS data , 2016, 1606.07987.

[2]  Frank Noé,et al.  Variational tensor approach for approximating the rare-event kinetics of macromolecular systems. , 2016, The Journal of chemical physics.

[3]  Rolf Dornberger,et al.  ulti-objective evolutionary algorithm for the optimization of noisy combustion problems , 2002 .

[4]  D. Kumar,et al.  Water Resources Systems Planning and Management , 2014 .

[5]  I. Mezić,et al.  Analysis of Fluid Flows via Spectral Properties of the Koopman Operator , 2013 .

[6]  Petros Koumoutsakos,et al.  Data-assisted reduced-order modeling of extreme events in complex dynamical systems , 2018, PloS one.

[7]  Eberhard Bodenschatz,et al.  A quantitative study of three-dimensional Lagrangian particle tracking algorithms , 2006 .

[8]  Hua Yang,et al.  PIV-DCNN: cascaded deep convolutional neural networks for particle image velocimetry , 2017, Experiments in Fluids.

[9]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[10]  Antonio Celani,et al.  Flow Navigation by Smart Microswimmers via Reinforcement Learning , 2017, Physical review letters.

[11]  Ye Li,et al.  Cellular neural network to detect spurious vectors in PIV data , 2003 .

[12]  G. Labonté,et al.  A new neural network for particle-tracking velocimetry , 1999 .

[13]  P. Moin,et al.  A dynamic subgrid‐scale eddy viscosity model , 1990 .

[14]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[15]  Petros Koumoutsakos,et al.  Efficient collective swimming by harnessing vortices through deep reinforcement learning , 2018, Proceedings of the National Academy of Sciences.

[16]  I. Mezić,et al.  Spectral analysis of nonlinear flows , 2009, Journal of Fluid Mechanics.

[17]  M. Mesbahi,et al.  Toward an Algorithmic Control Theory , 2017 .

[18]  Stephane Pierret,et al.  Turbomachinery Blade Design Using a Navier–Stokes Solver and Artificial Neural Network , 1998 .

[19]  Petros Koumoutsakos,et al.  C-start: optimal start of larval fish , 2012, Journal of Fluid Mechanics.

[20]  Hao Wu,et al.  VAMPnets for deep learning of molecular kinetics , 2017, Nature Communications.

[21]  Mark Tygert,et al.  A Randomized Algorithm for Principal Component Analysis , 2008, SIAM J. Matrix Anal. Appl..

[22]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[23]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[24]  C. Meneveau,et al.  Scale-Invariance and Turbulence Models for Large-Eddy Simulation , 2000 .

[25]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[26]  N. Phan-Thien,et al.  Neural-network-based approximations for solving partial differential equations , 1994 .

[27]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[28]  Nils Thürey,et al.  tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow , 2018, ACM Trans. Graph..

[29]  C. Papadimitriou,et al.  OPTIMAL SENSOR PLACEMENT FOR THE ESTIMATION OF TURBULENCE MODEL PARAMETERS IN CFD , 2015 .

[30]  H. Schaeffer,et al.  Learning partial differential equations via data discovery and sparse optimization , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[31]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[32]  Karthik Duraisamy,et al.  Machine Learning-augmented Predictive Modeling of Turbulent Separated Flows over Airfoils , 2016, ArXiv.

[33]  Babak Hejazialhosseini,et al.  Reinforcement Learning and Wavelet Adapted Vortex Methods for Simulations of Self-propelled Swimmers , 2014, SIAM J. Sci. Comput..

[34]  Jaideep Pathak,et al.  Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. , 2018, Physical review letters.

[35]  M. Yousuff Hussaini,et al.  A statistical learning strategy for closed-loop control of fluid flows , 2016, 1604.03392.

[36]  Jean Rabault,et al.  Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control , 2018, Journal of Fluid Mechanics.

[37]  Terrence J. Sejnowski,et al.  Glider soaring via reinforcement learning in the field , 2018, Nature.

[38]  Xi Chen,et al.  Evolution Strategies as a Scalable Alternative to Reinforcement Learning , 2017, ArXiv.

[39]  Sergios Theodoridis,et al.  Machine Learning: A Bayesian and Optimization Perspective , 2015 .

[40]  Martin Pelikan,et al.  Computational Complexity and Simulation of Rare Events of Ising Spin Glasses , 2004, GECCO.

[41]  Robert A. Lordo,et al.  Learning from Data: Concepts, Theory, and Methods , 2001, Technometrics.

[42]  Steven L. Brunton,et al.  Dynamic mode decomposition - data-driven modeling of complex systems , 2016 .

[43]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[44]  Ioannis G. Kevrekidis,et al.  DISCRETE- vs. CONTINUOUS-TIME NONLINEAR SIGNAL PROCESSING OF Cu ELECTRODISSOLUTION DATA , 1992 .

[45]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[46]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[47]  I. Grant,et al.  An investigation of the performance of multi layer, neural networks applied to the analysis of PIV images , 1995 .

[48]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[49]  Steven L. Brunton,et al.  Constrained sparse Galerkin regression , 2016, Journal of Fluid Mechanics.

[50]  William T. Freeman,et al.  Example-Based Super-Resolution , 2002, IEEE Computer Graphics and Applications.

[51]  P. Kutler,et al.  Computational aerodynamics and artificial intelligence , 1984 .

[52]  Dimitrios I. Fotiadis,et al.  Artificial neural networks for solving ordinary and partial differential equations , 1997, IEEE Trans. Neural Networks.

[53]  Karthik Duraisamy,et al.  A paradigm for data-driven predictive modeling using field inversion and machine learning , 2016, J. Comput. Phys..

[54]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[55]  Bernd R. Noack,et al.  Closed-Loop Turbulence Control-From Human to Machine Learning (and Retour) , 2017 .

[56]  Clarence W. Rowley,et al.  A Kernel Approach to Data-Driven Koopman Spectral Analysis , 2014 .

[57]  P. Koumoutsakos,et al.  1 Supplementary Information : Optimal morphokinematics for undulatory swimmers at intermediate Reynolds numbers , 2015 .

[58]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[59]  C. Bishop,et al.  Analysis of multiphase flows using dual-energy gamma densitometry and neural networks , 1993 .

[60]  Julia Ling,et al.  Machine learning strategies for systems with invariance properties , 2016, J. Comput. Phys..

[61]  Steven L. Brunton,et al.  Intracycle angular velocity control of cross-flow turbines , 2016, Nature Energy.

[62]  Yi Li,et al.  Data exploration of turbulence simulations using a database cluster , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[63]  E. Candès,et al.  Controlling the false discovery rate via knockoffs , 2014, 1404.5609.

[64]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[65]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[66]  Gautam Reddy,et al.  Learning to soar in turbulent environments , 2016, Proceedings of the National Academy of Sciences.

[67]  R. Goodman,et al.  Application of neural networks to turbulence control for drag reduction , 1997 .

[68]  Michael W. Mahoney.  Randomized Algorithms for Matrices and Data , 2011, Found. Trends Mach. Learn..

[69]  Bernd R. Noack,et al.  Cluster-based reduced-order modelling of a mixing layer , 2013, Journal of Fluid Mechanics.

[70]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[71]  Scott T. M. Dawson,et al.  Model Reduction for Flow Analysis and Control , 2017 .

[72]  Jürgen Schmidhuber,et al.  Multi-dimensional Recurrent Neural Networks , 2007, ICANN.

[73]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[74]  K. Taira,et al.  Super-resolution reconstruction of turbulent flows with machine learning , 2018, Journal of Fluid Mechanics.

[75]  Steven L. Brunton,et al.  Data-Driven Science and Engineering , 2019 .

[76]  K. Giannakoglou,et al.  Aerodynamic shape design using evolutionary algorithms and new gradient-assisted metamodels , 2006 .

[77]  Vassilios Theofilis,et al.  Modal Analysis of Fluid Flows: An Overview , 2017, 1702.01453.

[78]  Petros Koumoutsakos,et al.  Learning to school in the presence of hydrodynamic interactions , 2015, Journal of Fluid Mechanics.

[79]  Wei Hou,et al.  Machine Learning Based Detection of Flow Disturbances Using Surface Pressure Measurements , 2019, AIAA Scitech 2019 Forum.

[80]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[81]  Roger Temam,et al.  DNS-based predictive control of turbulence: an optimal benchmark for feedback algorithms , 2001, Journal of Fluid Mechanics.

[82]  Vladlen Koltun,et al.  Playing for Data: Ground Truth from Computer Games , 2016, ECCV.

[83]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[84]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[85]  Morteza Gharib,et al.  Experimental trajectory optimization of a flapping fin propulsor using an evolutionary strategy , 2018, Bioinspiration & biomimetics.

[86]  Petros Koumoutsakos,et al.  Learning probability distributions in continuous evolutionary algorithms – a comparative review , 2004, Natural Computing.

[87]  J. Periaux,et al.  Turbulent separated shear flow control by surface plasma actuator: experimental optimization by genetic algorithm approach , 2016 .

[88]  Dimitris C. Dracopoulos,et al.  Evolutionary Learning Algorithms for Neural Adaptive Control , 1997, Perspectives in Neural Computing.

[89]  Russ Tedrake.  Learning to Fly like a Bird , 2009 .

[90]  Diego Rossinelli,et al.  Synchronisation through learning for two self-propelled swimmers , 2015, Bioinspiration & biomimetics.

[91]  Lakshminarayanan Mahadevan,et al.  Controlled gliding and perching through deep-reinforcement-learning , 2019, Physical Review Fluids.

[92]  E. Gardner.  The space of interactions in neural network models , 1988 .

[93]  Peter J. Fleming,et al.  Evolutionary algorithms in control systems engineering: a survey , 2002 .

[94]  Mengying Wang,et al.  Detecting exotic wakes with hydrodynamic sensors , 2017, Theoretical and Computational Fluid Dynamics.

[95]  P. Moin,et al.  A direct numerical simulation study on the mean velocity characteristics in turbulent pipe flow , 2008, Journal of Fluid Mechanics.

[96]  C. L. Teo,et al.  A neural net approach in analyzing photograph in PIV , 1991, Conference Proceedings 1991 IEEE International Conference on Systems, Man, and Cybernetics.

[97]  Bryan Glaz,et al.  Reduced-Order Nonlinear Unsteady Aerodynamic Modeling Using a Surrogate-Based Recurrence Framework , 2010 .

[98]  Petros Koumoutsakos,et al.  A Method for Handling Uncertainty in Evolutionary Optimization With an Application to Feedback Control of Combustion , 2009, IEEE Transactions on Evolutionary Computation.

[99]  Stéphane Doncieux,et al.  Using Multiobjective Evolutionary Algorithms and Data-Mining Methods to Optimize Ornithopters' Kinematics , 2010 .

[100]  Steven L. Brunton,et al.  Data-Driven Sparse Sensor Placement , 2017, ArXiv.

[101]  L Sirovich,et al.  Low-dimensional procedure for the characterization of human faces. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[102]  E. Meijering.  A chronology of interpolation: from ancient astronomy to modern signal and image processing , 2002, Proc. IEEE.

[103]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[104]  Scott Schreck,et al.  Neural networks: Applications and opportunities in aeronautics , 1996 .

[105]  Frank Noé,et al.  Time-lagged autoencoders: Deep learning of slow collective variables for molecular kinetics , 2017, The Journal of chemical physics.

[106]  R Bellman,et al.  On the Theory of Dynamic Programming. , 1952, Proceedings of the National Academy of Sciences of the United States of America.

[107]  Petros Koumoutsakos,et al.  Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks , 2018, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[108]  Minh Q. Phan,et al.  On neural networks in identification and control of dynamic systems , 1993 .

[109]  N. Hansen,et al.  Step-Size Adaptation Based on Non-Local Use Selection Information , 1994 .

[110]  Hossein Zare-Behtash,et al.  State-of-the-art in aerodynamic shape optimisation methods , 2018, Appl. Soft Comput..

[111]  K. Taira,et al.  Network community-based model reduction for vortical flows. , 2018, Physical review. E.

[112]  Gerhardt von Bonin,et al.  Cybernetics or control and communication in the animal and the machine: Norbert wiener, 1948. 194 pp. New York: John Wiley & Sons, Inc. Paris: Hermann et cie , 1949 .

[113]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[114]  Peter Jordan,et al.  Qualitative dynamics of wavepackets in turbulent jets , 2016 .

[115]  Frank Noé,et al.  A Variational Approach to Modeling Slow Processes in Stochastic Dynamical Systems , 2012, Multiscale Model. Simul..

[116]  P. Schmid,et al.  Dynamic mode decomposition of numerical and experimental data , 2008, Journal of Fluid Mechanics.

[117]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[118]  Ioannis G Kevrekidis,et al.  Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator. , 2017, Chaos.

[119]  J. Kutz,et al.  Compressive Sensing Based Machine Learning Strategy For Characterizing The Flow Around A Cylinder With Limited Pressure Measurements , 2013 .

[120]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[121]  Luca Biferale,et al.  Smart inertial particles , 2017, Physical Review Fluids.

[122]  Heng Xiao,et al.  Quantifying and reducing model-form uncertainties in Reynolds-averaged Navier-Stokes simulations: A data-driven, physics-informed Bayesian approach , 2015, J. Comput. Phys..

[123]  Naoya Takeishi,et al.  Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition , 2017, NIPS.

[124]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[125]  Steven L. Brunton,et al.  Machine Learning Control – Taming Nonlinear Dynamics and Turbulence , 2016, Fluid Mechanics and Its Applications.

[126]  B. R. Noack,et al.  Closed-Loop Turbulence Control: Progress and Challenges , 2015 .

[127]  J. Templeton.  Evaluation of machine learning algorithms for prediction of regions of high Reynolds averaged Navier Stokes uncertainty , 2015 .

[128]  Karthik Duraisamy,et al.  Turbulence Modeling in the Age of Data , 2018, Annual Review of Fluid Mechanics.

[129]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[130]  Kunihiko Taira,et al.  Network-theoretic approach to sparsified discrete vortex dynamics , 2015, Journal of Fluid Mechanics.

[131]  M. Wernet.  Digital Particle Image Velocimetry , 2003 .

[132]  Ioannis G. Kevrekidis,et al.  Identification of distributed parameter systems: A neural net based approach , 1998 .

[133]  Gerald Tesauro,et al.  Practical issues in temporal difference learning , 1992, Machine Learning.

[134]  Daniele Venturi,et al.  Multifidelity Information Fusion Algorithms for High-Dimensional Systems and Massive Data sets , 2016, SIAM J. Sci. Comput..

[135]  Anne Auger,et al.  Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles , 2011, J. Mach. Learn. Res..

[136]  Hao Wu,et al.  Deep Generative Markov State Models , 2018, NeurIPS.

[137]  J. Templeton,et al.  Reynolds averaged turbulence modelling using deep neural networks with embedded invariance , 2016, Journal of Fluid Mechanics.

[138]  S. Shankar Sastry,et al.  Autonomous Helicopter Flight via Reinforcement Learning , 2003, NIPS.

[139]  Reinhold Orglmeister,et al.  A Hopfield neural network for flow field computation based on particle image velocimetry/particle tracking velocimetry image sequences , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[140]  S. Ashforth-Frost,et al.  Evaluating convective heat transfer coefficients using neural networks , 1996 .

[141]  Michele Milano,et al.  Neural network modeling for near wall turbulent flow , 2002 .

[142]  Prakash Vedula,et al.  Subgrid modelling for two-dimensional turbulence using neural networks , 2018, Journal of Fluid Mechanics.

[143]  A. Kolmogorov.  The local structure of turbulence in incompressible viscous fluid for very large Reynolds numbers , 1991, Proceedings of the Royal Society of London. Series A: Mathematical and Physical Sciences.