A Deep Learning based Approach to Reduced Order Modeling for Turbulent Flow Control using LSTM Neural Networks

Reduced Order Modeling (ROM) for engineering applications has been a major research focus in the past few decades due to the unprecedented physical insight into turbulence offered by high-fidelity CFD. The primary goal of a ROM is to model the key physics/features of a flow-field without computing the full Navier-Stokes (NS) equations. This is accomplished by projecting the high-dimensional dynamics to a low-dimensional subspace, typically utilizing dimensionality reduction techniques like Proper Orthogonal Decomposition (POD), coupled with Galerkin projection. In this work, we demonstrate a deep learning based approach to build a ROM using the POD basis of canonical DNS datasets, for turbulent flow control applications. We find that a type of Recurrent Neural Network, the Long Short Term Memory (LSTM) which has been primarily utilized for problems like speech modeling and language translation, shows attractive potential in modeling temporal dynamics of turbulence. Additionally, we introduce the Hurst Exponent as a tool to study LSTM behavior for non-stationary data, and uncover useful characteristics that may aid ROM development for a variety of applications.

[1]  Claude E. Shannon,et al.  Prediction and Entropy of Printed English , 1951 .

[2]  H. E. Hurst,et al.  Long-Term Storage Capacity of Reservoirs , 1951 .

[3]  J. R. Wallis,et al.  Noah, Joseph, and Operational Hydrology , 1968 .

[4]  A. G. Ivakhnenko,et al.  Polynomial Theory of Complex Systems , 1971, IEEE Trans. Syst. Man Cybern..

[5]  G. S. Patterson,et al.  Spectral Calculations of Isotropic Turbulence: Efficient Removal of Aliasing Interactions , 1971 .

[6]  Wentian Li,et al.  Random texts exhibit Zipf's-law-like word frequency distribution , 1992, IEEE Trans. Inf. Theory.

[7]  P. Holmes,et al.  The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows , 1993 .

[8]  P. Holmes,et al.  Turbulence, Coherent Structures, Dynamical Systems and Symmetry , 1996 .

[9]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[10]  S. Ravindran,et al.  A Reduced-Order Method for Simulation and Control of Fluid Flows , 1998 .

[11]  Sepp Hochreiter,et al.  The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions , 1998, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[12]  Jean-Antoine Désidéri,et al.  Stability Properties of POD–Galerkin Approximations for the Compressible Navier–Stokes Equations , 2000 .

[13]  D. Rempfer,et al.  On Low-Dimensional Galerkin Models for Fluid Flow , 2000 .

[14]  S. Ravindran A reduced-order approach for optimal control of fluids using proper orthogonal decomposition , 2000 .

[15]  J. Sprott Chaos and time-series analysis , 2001 .

[16]  George Carayannis,et al.  Word Length, Word Frequencies and Zipf’s Law in the Greek Language , 2001, J. Quant. Linguistics.

[17]  Stephan Roche,et al.  Long range correlations in DNA: scaling properties and charge transfer efficiency. , 2003, Physical review letters.

[18]  Demetris Koutsoyiannis,et al.  Climate change, the Hurst phenomenon, and hydrological statistics , 2003 .

[19]  R. Murray,et al.  Model reduction for compressible flows using POD and Galerkin projection , 2004 .

[20]  H. Stanley,et al.  Time-dependent Hurst exponent in financial time series , 2004 .

[21]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[22]  Jürgen Schmidhuber,et al.  Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition , 2005, ICANN.

[23]  Max Gunzburger,et al.  POD and CVT-based reduced-order modeling of Navier-Stokes flows , 2006 .

[24]  Clarence W. Rowley,et al.  Dynamics and control of high-reynolds-number flow over open cavities , 2006 .

[25]  Rob J Hyndman,et al.  Another look at measures of forecast accuracy , 2006 .

[26]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[27]  Matthew F. Barone,et al.  Stable Galerkin reduced order models for linearized compressible flow , 2009, J. Comput. Phys..

[28]  Ali H. Nayfeh,et al.  On the stability and extension of reduced-order Galerkin models in incompressible flows , 2009 .

[29]  José M. Vega,et al.  Reduced order models based on local POD plus Galerkin projection , 2010, J. Comput. Phys..

[30]  C. Farhat,et al.  Efficient non‐linear model reduction via a least‐squares Petrov–Galerkin projection and compressive tensor approximations , 2011 .

[31]  R. Weron HURST: MATLAB function to compute the Hurst exponent using R/S Analysis , 2011 .

[32]  Mark Pagel,et al.  How do we use language? Shared patterns in the frequency of word use across 17 world languages , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[33]  Jiansheng Wu,et al.  Trend analysis of vegetation dynamics in Qinghai–Tibet Plateau using Hurst Exponent , 2012 .

[34]  Gilead Tadmor,et al.  Reduced-Order Modelling for Flow Control , 2013 .

[35]  Bernd R. Noack,et al.  Cluster-based reduced-order modelling of a mixing layer , 2013, Journal of Fluid Mechanics.

[36]  Frank K. Soong,et al.  TTS synthesis with bidirectional LSTM based recurrent neural networks , 2014, INTERSPEECH.

[37]  Erik Marchi,et al.  Multi-resolution linear prediction based features for audio onset detection with bidirectional LSTM neural networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[38]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[39]  J. Templeton Evaluation of machine learning algorithms for prediction of regions of high Reynolds averaged Navier Stokes uncertainty , 2015 .

[40]  Anand Pratap Singh,et al.  New Approaches in Turbulence and Transition Modeling Using Data-driven Techniques , 2015 .

[41]  Wei Xu,et al.  Bidirectional LSTM-CRF Models for Sequence Tagging , 2015, ArXiv.

[42]  Myoungkyu Lee,et al.  A Web services accessible database of turbulent channel flow and its use for testing a new integral wall model for LES , 2016 .

[43]  Heng Xiao,et al.  Physics-Informed Machine Learning for Predictive Turbulence Modeling: Using Data to Improve RANS Modeled Reynolds Stresses , 2016 .

[44]  Steven L. Brunton,et al.  Machine Learning Control – Taming Nonlinear Dynamics and Turbulence , 2016, Fluid Mechanics and Its Applications.

[45]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[46]  Elisabeth Longatte,et al.  A Galerkin-free model reduction approach for the Navier-Stokes equations , 2016, J. Comput. Phys..

[47]  Jinlong Wu,et al.  Physics-informed machine learning approach for reconstructing Reynolds stress modeling discrepancies based on DNS data , 2016, 1606.07987.

[48]  C. Pain,et al.  Model identification of reduced order fluid dynamics systems using deep learning , 2017, International Journal for Numerical Methods in Fluids.