A comparison of combined data assimilation and machine learning methods for offline and online model error correction

Recent studies have shown that it is possible to combine machine learning methods with data assimilation to reconstruct a dynamical system using only sparse and noisy observations of that system. The same approach can be used to correct the error of a knowledge-based model. The resulting surrogate model is hybrid, with a statistical part supplementing a physical part. In practice, the correction can be added as an integrated term (i.e. in the model resolvent) or directly inside the tendencies of the physical model. The resolvent correction is easy to implement. The tendency correction is more technical, in particular it requires the adjoint of the physical model, but also more flexible. We use the two-scale Lorenz model to compare the two methods. The accuracy in long-range forecast experiments is somewhat similar between the surrogate models using the resolvent correction and the tendency correction. By contrast, the surrogate models using the tendency correction significantly outperform the surrogate models using the resolvent correction in data assimilation experiments. Finally, we show that the tendency correction opens the possibility to make online model error correction, i.e. improving the model progressively as new observations become available. The resulting algorithm can be seen as a new formulation of weak-constraint 4D-Var. We compare online and offline learning using the same framework with the two-scale Lorenz system, and show that with online learning, it is possible to extract all the information from sparse and noisy observations.

[1]  Alberto Carrassi,et al.  Accounting for model error due to unresolved scales within ensemble Kalman filtering , 2014, 1409.0589.

[2]  Redouane Lguensat,et al.  The Analog Data Assimilation , 2017 .

[3]  Pierre Gentine,et al.  Deep learning to represent subgrid processes in climate models , 2018, Proceedings of the National Academy of Sciences.

[4]  Takemasa Miyoshi,et al.  Estimating Model Parameters with Ensemble-Based Data Assimilation: A Review , 2013 .

[5]  Y. Trémolet.  Accounting for an imperfect model in 4D‐Var , 2006 .

[6]  Sebastian Reich,et al.  Supervised learning from noisy observations: Combining machine-learning techniques with data assimilation , 2020, ArXiv.

[7]  Jaideep Pathak,et al.  Model-Free Prediction of Large Spatiotemporally Chaotic Systems from Data: A Reservoir Computing Approach. , 2018, Physical review letters.

[8]  Marc Bocquet,et al.  Combining data assimilation and machine learning to infer unresolved scale parametrization , 2020, Philosophical Transactions of the Royal Society A.

[9]  Marc Bocquet,et al.  Online learning of both state and dynamics using ensemble Kalman filters , 2020, ArXiv.

[10]  Hannah M. Christensen,et al.  Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model , 2019, Journal of Advances in Modeling Earth Systems.

[11]  Cédric Herzet,et al.  Bilinear Residual Neural Network for the Identification and Forecasting of Geophysical Dynamics , 2018, 2018 26th European Signal Processing Conference (EUSIPCO).

[12]  Marc Bocquet,et al.  State, global and local parameter estimation using local ensemble Kalman filters: applications to online machine learning of chaotic dynamics , 2021, ArXiv.

[13]  Peter A. G. Watson,et al.  Applying Machine Learning to Improve Simulations of a Chaotic Dynamical System Using Empirical Error Correction , 2019, Journal of advances in modeling earth systems.

[14]  William W. Hsieh,et al.  Applying Neural Network Models to Prediction and Data Analysis in Meteorology and Oceanography. , 1998 .

[15]  K. Emanuel,et al.  Optimal Sites for Supplementary Weather Observations: Simulation with a Small Model , 1998 .

[16]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[17]  Henry D. I. Abarbanel,et al.  Machine Learning: Deepest Learning as Statistical Data Assimilation Problems , 2017, Neural Computation.

[18]  Marc Bocquet,et al.  Data assimilation as a learning tool to infer ordinary differential equation representations of dynamical models , 2019, Nonlinear Processes in Geophysics.

[19]  Olivier Talagrand,et al.  On extending the limits of variational assimilation in nonlinear chaotic systems , 1996 .

[20]  Anuj Karpatne,et al.  Physics Guided RNNs for Modeling Dynamical Systems: A Case Study in Simulating Lake Temperature Profiles , 2018, SDM.

[21]  D. Wilks.  Effects of stochastic parametrizations in the Lorenz '96 system , 2005 .

[22]  Alberto Carrassi,et al.  Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods , 2017, 1709.07328.

[23]  S. Brunton,et al.  Discovering governing equations from data by sparse identification of nonlinear dynamical systems , 2015, Proceedings of the National Academy of Sciences.

[24]  Peter Bauer,et al.  Challenges and design choices for global weather and climate models based on machine learning , 2018, Geoscientific Model Development.

[25]  Serge Gratton,et al.  Quasi-static ensemble variational data assimilation: a theoretical and numerical study with the iterative ensemble Kalman smoother , 2018 .

[26]  Tyrus Berry,et al.  Ensemble Kalman Filtering without a Model , 2016 .

[27]  Nikhil Ketkar,et al.  Deep Learning with Python , 2017 .

[28]  Marcin Chrust,et al.  Exploring the potential and limitations of weak‐constraint 4D‐Var , 2020, Quarterly Journal of the Royal Meteorological Society.

[29]  Garrison W. Cottrell,et al.  ReZero is All You Need: Fast Convergence at Large Depth , 2020, UAI.

[30]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[31]  Massimo Bonavita,et al.  Using machine learning to correct model error in data assimilation and forecast applications , 2020, Quarterly Journal of the Royal Meteorological Society.

[32]  Jonathan A. Weyn,et al.  Can Machines Learn to Predict Weather? Using Deep Learning to Predict Gridded 500‐hPa Geopotential Height From Historical Weather Data , 2019, Journal of Advances in Modeling Earth Systems.

[33]  Marc Bocquet,et al.  Combining data assimilation and machine learning to emulate a dynamical model from sparse and noisy observations: A case study with the Lorenz 96 model , 2020, J. Comput. Sci..

[34]  Patrick Laloyaux,et al.  Machine Learning for Model Error Inference and Correction , 2020 .

[35]  J. Thepaut,et al.  The ERA5 global reanalysis , 2020, Quarterly Journal of the Royal Meteorological Society.

[36]  S. Healy,et al.  Towards an unbiased stratospheric analysis , 2020, Quarterly Journal of the Royal Meteorological Society.

[37]  Sebastian Scher,et al.  Generalization properties of feed-forward neural networks trained on Lorenz systems , 2019, Nonlinear Processes in Geophysics.

[38]  R. Arcucci,et al.  Deep Data Assimilation: Integrating Deep Learning with Data Assimilation , 2021, Applied Sciences.

[39]  Edward N. Lorenz,et al.  Designing Chaotic Models , 2005 .

[40]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[41]  Thomas Bolton,et al.  Applications of Deep Learning to Ocean Data Inference and Subgrid Parameterization , 2019, Journal of Advances in Modeling Earth Systems.

[42]  Jaideep Pathak,et al.  Combining Machine Learning with Knowledge-Based Modeling for Scalable Forecasting and Subgrid-Scale Closure of Large, Complex, Spatiotemporal Systems , 2020, Chaos.

[43]  Jaideep Pathak,et al.  A Machine Learning‐Based Global Atmospheric Forecast Model , 2020 .

[44]  Florian Nadel,et al.  Stochastic Processes And Filtering Theory , 2016 .