Learning Generalized Quasi-Geostrophic Models Using Deep Neural Numerical Models

We introduce a new strategy designed to help physicists discover hidden laws governing dynamical systems. We propose to use machine learning automatic differentiation libraries to develop hybrid numerical models that combine components based on prior physical knowledge with components based on neural networks. In these architectures, named Deep Neural Numerical Models (DNNMs), the neural network components are used as building-blocks then deployed for learning hidden variables of underlying physical laws governing dynamical systems. In this paper, we illustrate an application of DNNMs to upper ocean dynamics, more precisely the dynamics of a sea surface tracer, the Sea Surface Height (SSH). We develop an advection-based fully differentiable numerical scheme, where parts of the computations can be replaced with learnable ConvNets, and make connections with the single-layer Quasi-Geostrophic (QG) model, a baseline theory in physical oceanography developed decades ago.

[1]  Max Tegmark,et al.  Toward an artificial intelligence physicist for unsupervised learning. , 2019, Physical review. E.

[2]  G. Flierl,et al.  Nonlinear energy and enstrophy transfers in a realistically stratified ocean , 1980 .

[3]  Patrick Gallinari,et al.  Deep learning for physical processes: incorporating prior scientific knowledge , 2017, ICLR.

[4]  Marin Soljacic,et al.  Extracting Interpretable Physical Parameters from Spatiotemporal Systems using Unsupervised Learning , 2019, Physical Review X.

[5]  Max Tegmark,et al.  Toward an AI Physicist for Unsupervised Learning , 2018, Physical review. E.

[6]  Ronan Fablet,et al.  Residual Networks as Flows of Diffeomorphisms , 2019, Journal of Mathematical Imaging and Vision.

[7]  Bertrand Chapron,et al.  Learning Latent Dynamics for Partially-Observed Chaotic Systems , 2019, Chaos.

[8]  Nathan R. Tallent,et al.  ADJOINT CODE BY SOURCE TRANSFORMATION WITH OPENAD/F , 2006 .

[9]  D. Haidvogel,et al.  Numerical Simulations of the Vertical Structure of Quasi-Geostrophic Turbulence , 1986 .

[10]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[11]  Jason Yosinski,et al.  Hamiltonian Neural Networks , 2019, NeurIPS.

[12]  E Weinan,et al.  A Proposal on Machine Learning via Dynamical Systems , 2017, Communications in Mathematics and Statistics.

[13]  Naftali Tishby,et al.  Machine learning and the physical sciences , 2019, Reviews of Modern Physics.

[14]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[15]  Geoffrey K. Vallis,et al.  The scales and equilibration of midocean eddies : Freely evolving flow , 2001 .

[16]  Patrick Gallinari,et al.  Learning Dynamical Systems from Partial Observations , 2019, ArXiv.

[17]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[18]  Lee-Lueng Fu,et al.  Dynamic Interpolation of Sea Surface Height and Potential Applications for Future High-Resolution Altimetry Mapping , 2015 .

[19]  Ce Wang,et al.  Emergent Schrödinger equation in an introspective machine learning architecture , 2019 .

[20]  Maziar Raissi,et al.  Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations , 2018, J. Mach. Learn. Res..

[21]  Barak A. Pearlmutter,et al.  Automatic differentiation in machine learning: a survey , 2015, J. Mach. Learn. Res..

[22]  J. Shewchuk.  An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[23]  Renato Renner,et al.  Discovering physical concepts with neural networks , 2018, Physical review letters.

[24]  G. Vallis.  Atmospheric and Oceanic Fluid Dynamics: Fundamentals and Large-Scale Circulation , 2017 .

[25]  Jean Utke,et al.  OpenAD/F: A Modular Open-Source Tool for Automatic Differentiation of Fortran Codes , 2008, TOMS.

[26]  Bin Dong,et al.  PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network , 2018, J. Comput. Phys..