Bridging the Gap: Machine Learning to Resolve Improperly Modeled Dynamics

Abstract We present a data-driven modeling strategy to overcome improperly modeled dynamics for systems exhibiting complex spatio-temporal behaviors. We propose a Deep Learning framework to resolve the differences between the true dynamics of the system and the dynamics given by a model of the system that is either inaccurately or inadequately described. Our machine learning strategy leverages data generated from the improper system model and observational data from the actual system to create a neural network to model the dynamics of the actual system. We evaluate the proposed framework using numerical solutions obtained from three increasingly complex dynamical systems. Our results show that our system is capable of learning a data-driven model that provides accurate estimates of the system states both in previously unobserved regions as well as for future states. Our results show the power of state-of-the-art machine learning frameworks in estimating an accurate prior of the system’s true dynamics that can be used for prediction up to a finite horizon.

[1]  Stephen Marshall,et al.  Activation Functions: Comparison of trends in Practice and Research for Deep Learning , 2018, ArXiv.

[2]  Michelle Girvan,et al.  Hybrid Forecasting of Chaotic Processes: Using Machine Learning in Conjunction with a Knowledge-Based Model , 2018, Chaos.

[3]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[4]  Donghyun You,et al.  Prediction of laminar vortex shedding over a cylinder using deep learning , 2017 .

[5]  Yachen Lin,et al.  Geometric Data Analysis: An Empirical Approach to Dimensionality Reduction and the Study of Patterns , 2002, Technometrics.

[6]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Stefan Roth,et al.  Benchmarking Denoising Algorithms with Real Photographs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Anuj Karpatne,et al.  Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling , 2017, ArXiv.

[10]  T. Poinsot,et al.  Training convolutional neural networks to estimate turbulent sub-grid scale reaction rates , 2018, Combustion and Flame.

[11]  Guy-Bart Stan,et al.  A Sparse Bayesian Approach to the Identification of Nonlinear State-Space Systems , 2014, IEEE Transactions on Automatic Control.

[12]  T. P. Miyanawala,et al.  An Efficient Deep Learning Technique for the Navier-Stokes Equations: Application to Unsteady Wake Flow Dynamics , 2017, 1710.09099.

[13]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations , 2017, ArXiv.

[14]  Nils Thürey,et al.  Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow , 2018, Comput. Graph. Forum.

[15]  Ed H. Chi,et al.  AntisymmetricRNN: A Dynamical System View on Recurrent Neural Networks , 2019, ICLR.

[16]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[17]  J. Templeton.  Evaluation of machine learning algorithms for prediction of regions of high Reynolds averaged Navier Stokes uncertainty , 2015 .

[18]  A. Mohan,et al.  A Deep Learning based Approach to Reduced Order Modeling for Turbulent Flow Control using LSTM Neural Networks , 2018, 1804.09269.

[19]  Erik M. Bollt,et al.  Modeling and nonlinear parameter estimation with Kronecker product representation for coupled oscillators and spatiotemporal systems , 2007 .

[20]  G. Karniadakis,et al.  Multistep Neural Networks for Data-driven Discovery of Nonlinear Dynamical Systems , 2018, 1801.01236.

[21]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[22]  Zoubin Ghahramani,et al.  Probabilistic machine learning and artificial intelligence , 2015, Nature.

[23]  Patrick Gallinari,et al.  Learning Dynamical Systems from Partial Observations , 2019, ArXiv.

[24]  Alex Sherstinsky,et al.  Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network , 2018, Physica D: Nonlinear Phenomena.

[25]  L. Sirovich.  Turbulence and the dynamics of coherent structures. II. Symmetries and transformations , 1987 .

[26]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[27]  Elie Hachem,et al.  A supervised neural network for drag prediction of arbitrary 2D shapes in low Reynolds number flows , 2019 .

[28]  Paris Perdikaris,et al.  Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations , 2017, ArXiv.

[29]  Erik Bollt,et al.  How Entropic Regression Beats the Outliers Problem in Nonlinear System Identification , 2019, Chaos.

[30]  Jonathan D. Rogers,et al.  Causation Entropy Identifies Sparsity Structure for Parameter Estimation of Dynamic Systems , 2017 .

[31]  Prakash Vedula,et al.  Data-driven deconvolution for large eddy simulations of Kraichnan turbulence , 2018, Physics of Fluids.

[32]  B. Frey,et al.  Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning , 2015, Nature Biotechnology.

[33]  L. Sirovich.  Turbulence and the dynamics of coherent structures. I. Coherent structures , 1987 .

[34]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[35]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[36]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[37]  Hod Lipson,et al.  Automated reverse engineering of nonlinear dynamical systems , 2007, Proceedings of the National Academy of Sciences.

[38]  Wojciech Zaremba,et al.  Deep Neural Networks Predict Category Typicality Ratings for Images , 2015, CogSci.

[39]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[40]  Prakash Vedula,et al.  Subgrid modelling for two-dimensional turbulence using neural networks , 2018, Journal of Fluid Mechanics.

[41]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[42]  Karthik Duraisamy,et al.  Prediction of aerodynamic flow fields using convolutional neural networks , 2019, Computational Mechanics.

[43]  George E. Karniadakis,et al.  Hidden Fluid Mechanics: A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data , 2018, ArXiv.