Learning Dynamical Systems from Partial Observations

We consider the problem of forecasting complex, nonlinear space-time processes when observations provide only partial information of on the system's state. We propose a natural data-driven framework, where the system's dynamics are modelled by an unknown time-varying differential equation, and the evolution term is estimated from the data, using a neural network. Any future state can then be computed by placing the associated differential equation in an ODE solver. We first evaluate our approach on shallow water and Euler simulations. We find that our method not only demonstrates high quality long-term forecasts, but also learns to produce hidden states closely resembling the true states of the system, without direct supervision on the latter. Additional experiments conducted on challenging, state of the art ocean simulations further validate our findings, while exhibiting notable improvements over classical baselines.

[1]  Paris Perdikaris,et al.  Machine learning of linear differential equations using Gaussian processes , 2017, J. Comput. Phys..

[2]  Max Gunzburger,et al.  Perspectives in flow control and optimization , 1987 .

[3]  Patrick Gallinari,et al.  Deep learning for physical processes: incorporating prior scientific knowledge , 2017, ICLR.

[4]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[5]  Steven L. Brunton,et al.  Data-driven discovery of partial differential equations , 2016, Science Advances.

[6]  G. Madec NEMO ocean engine , 2008 .

[7]  Marc Bocquet,et al.  Parameter‐field estimation for atmospheric dispersion: application to the Chernobyl accident using 4D‐Var , 2012 .

[8]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[9]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[10]  E. Tziperman,et al.  Finite Difference of Adjoint or Adjoint of Finite Difference , 1997 .

[11]  Dit-Yan Yeung,et al.  Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting , 2015, NIPS.

[12]  Andrew C. Lorenc,et al.  Analysis methods for numerical weather prediction , 1986 .

[13]  Cédric Herzet,et al.  Bilinear Residual Neural Network for the Identification and Forecasting of Geophysical Dynamics , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  James P. Crutchfield,et al.  Equations of Motion from a Data Series , 1987, Complex Syst..

[16]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[17]  Isabelle Herlin,et al.  Coupling Dynamic Equations and Satellite Images for Modelling Ocean Surface Circulation , 2014, VISIGRAPP.

[18]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[19]  E Weinan,et al.  A Proposal on Machine Learning via Dynamical Systems , 2017, Communications in Mathematics and Statistics.

[20]  Yann Le Cun,et al.  A Theoretical Framework for Back-Propagation , 1988 .

[21]  D. Schaeffer,et al.  A two-current model for the dynamics of cardiac membrane , 2003, Bulletin of mathematical biology.

[22]  Alex Graves,et al.  Video Pixel Networks , 2016, ICML.

[23]  Philip S. Yu,et al.  PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning , 2018, ICML.

[24]  John Ferguson Geological applications of differential equations , 1988 .

[25]  Cecile Cabanes,et al.  Global Eddy-Permitting Ocean Reanalyses and Simulations of the Period 1992 to Present , 2013 .

[26]  Cédric Herzet,et al.  Bilinear residual Neural Network for the identification and forecasting of dynamical systems , 2017, ArXiv.

[27]  G. Evensen,et al.  Data assimilation in the geosciences: An overview of methods, issues, and perspectives , 2017, WIREs Climate Change.

[28]  Yves Achdou,et al.  Partial Differential Equations in Finance , 2012 .

[29]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[30]  Ludovic Denoyer,et al.  Spatio-Temporal Neural Networks for Space-Time Series Forecasting and Relations Discovery , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[31]  Patrick Gallinari,et al.  Optimal Unsupervised Domain Translation , 2019, ArXiv.