Current and Emerging Time-Integration Strategies in Global Numerical Weather and Climate Prediction

AbstractThe continuous partial differential equations governing a given physical phenomenon, such as the Navier–Stokes equations describing the fluid motion, must be numerically discretized in space and time in order to obtain a solution otherwise not readily available in closed (i.e., analytic) form. While the overall numerical discretization plays an essential role in the algorithmic efficiency and physically-faithful representation of the solution, the time-integration strategy commonly is one of the main drivers in terms of cost-to-solution (e.g., time- or energy-to-solution), accuracy and numerical stability, thus constituting one of the key building blocks of the computational model. This is especially true in time-critical applications, including numerical weather prediction (NWP), climate simulations and engineering. This review provides a comprehensive overview of the existing and emerging time-integration (also referred to as time-stepping) practices used in the operational global NWP and climate industry, where global refers to weather and climate simulations performed on the entire globe. While there are many flavors of time-integration strategies, in this review we focus on the most widely adopted in NWP and climate centers and we emphasize the reasons why such numerical solutions were adopted. This allows us to make some considerations on future trends in the field such as the need to balance accuracy in time with substantially enhanced time-to-solution and associated implications on energy consumption and running costs. In addition, the potential for the co-design of time-stepping algorithms and underlying high performance computing hardware, a keystone to accelerate the computational performance of future NWP and climate services, is also discussed in the context of the demanding operational requirements of the weather and climate industry.

[1]  Lloyd N. Trefethen,et al.  Fourth-Order Time-Stepping for Stiff PDEs , 2005, SIAM J. Sci. Comput..

[2]  André Robert,et al.  Spurious Resonant Response of Semi-Lagrangian Discretizations to Orographic Forcing: Diagnosis and Solution , 1994 .

[3]  Hester Bijl,et al.  Fourth-Order Runge–Kutta Schemes for Fluid Mechanics Applications , 2005, J. Sci. Comput..

[4]  Günther Zängl,et al.  Extending the Numerical Stability Limit of Terrain-Following Coordinate Models over Steep Slopes , 2012 .

[5]  J. R. Cash,et al.  Diagonally Implicit Runge-Kutta Formulae with Error Estimates , 1979 .

[6]  Harold Ritchie,et al.  Semi-Lagrangian advection on a Gaussian grid , 1987 .

[7]  T. E. Hull,et al.  Numerical solution of initial value problems , 1966 .

[8]  Rupert Klein,et al.  A Blended Soundproof-to-Compressible Numerical Model for Small- to Mesoscale Atmospheric Dynamics , 2014 .

[9]  Mats Hamrud,et al.  A Fast Spherical Harmonics Transform for Global NWP and Climate Models , 2013 .

[10]  John C. Butcher.  General linear methods , 2006, Acta Numerica.

[11]  A. Simmons,et al.  Implementation of the Semi-Lagrangian Method in a High-Resolution Version of the ECMWF Forecast Model , 1995 .

[12]  A. Arakawa,et al.  Unification of the Anelastic and Quasi-Hydrostatic Systems of Equations , 2009 .

[13]  René Laprise,et al.  The Euler Equations of Motion with Hydrostatic Pressure as an Independent Variable , 1992 .

[14]  Spencer J. Sherwin,et al.  A generic framework for time-stepping partial differential equations (PDEs): general linear methods, object-oriented implementation and application to fluid problems , 2011 .

[15]  F. Giraldo.  Lagrange-Galerkin Methods on Spherical Geodesic Grids , 1997 .

[16]  Nigel Wood,et al.  SLICE: A Semi‐Lagrangian Inherently Conserving and Efficient scheme for transport problems , 2002 .

[17]  Peter Bauer,et al.  The quiet revolution of numerical weather prediction , 2015, Nature.

[18]  Jimy Dudhia,et al.  Conservative Split-Explicit Time Integration Methods for the Compressible Nonhydrostatic Equations , 2007 .

[19]  Jean Côté,et al.  The CMC-MRB Global Environmental Multiscale (GEM) Model. Part III: Nonhydrostatic Formulation , 2002 .

[20]  Piet Termonia,et al.  Discretization in Numerical Weather Prediction: A Modular Approach to Investigate Spectral and Local SISL Methods , 2016 .

[21]  Jeffrey S. Scroggs,et al.  A global nonhydrostatic semi-Lagrangian atmospheric model without orography , 1995 .

[22]  Paul A. Ullrich,et al.  A conservative semi-Lagrangian multi-tracer transport scheme (CSLAM) on the cubed-sphere grid , 2010, J. Comput. Phys..

[23]  P. Smolarkiewicz.  Modeling atmospheric circulations with soundproof equations , 2008 .

[24]  John C. Butcher,et al.  General linear methods for ordinary differential equations , 2009, Math. Comput. Simul..

[25]  Constantine Bekas,et al.  An extreme-scale implicit solver for complex PDEs: highly heterogeneous flow in earth's mantle , 2015, SC15: International Conference for High Performance Computing, Networking, Storage and Analysis.

[26]  A. Kværnø,et al.  Norges Teknisk-naturvitenskapelige Universitet Singly Diagonally Implicit Runge-kutta Methods with an Explicit First Stage Singly Diagonally Implicit Runge-kutta Methods with an Explicit First Stage , 2022 .

[27]  N. Jeevanjee.  Vertical Velocity in the Gray Zone , 2016 .

[28]  Sascha M. Schnepp,et al.  Pipelined, Flexible Krylov Subspace Methods , 2015, SIAM J. Sci. Comput..

[29]  Piotr K. Smolarkiewicz,et al.  Predicting weather, climate and extreme events , 2008, J. Comput. Phys..

[30]  Jeff Cash,et al.  On the integration of stiff systems of O.D.E.s using extended backward differentiation formulae , 1980 .

[31]  Pierre Bénard,et al.  Dynamical kernel of the Aladin–NH spectral limited‐area model: Revised formulation and sensitivity experiments , 2010 .

[32]  R. Brayton,et al.  A new efficient algorithm for solving differential-algebraic systems using implicit backward differentiation formulas , 1972 .

[33]  Christopher A. Kennedy,et al.  Diagonally implicit Runge–Kutta methods for stiff ODEs , 2019 .

[34]  Jeff Cash,et al.  The integration of stiff initial value problems in ODEs using modified extended backward differentiation formulae , 1983 .

[35]  M. Falcone,et al.  Convergence Analysis for a Class of High-Order Semi-Lagrangian Advection Schemes , 1998 .

[36]  Dale R. Durran,et al.  The Third-Order Adams-Bashforth Method: An Attractive Alternative to Leapfrog Time Differencing , 1991 .

[37]  George Em Karniadakis,et al.  A semi-Lagrangian high-order method for Navier-Stokes equations , 2001 .

[38]  Nils Wedi,et al.  A framework for testing global non‐hydrostatic models , 2009 .

[39]  D. Williamson.  The Evolution of Dynamical Cores for Global Atmospheric Models(125th Anniversary Issue of the Meteorological Society of Japan) , 2007 .

[40]  Francis X. Giraldo,et al.  A study of spectral element and discontinuous Galerkin methods for the Navier-Stokes equations in nonhydrostatic mesoscale atmospheric modeling: Equation sets and test cases , 2008, J. Comput. Phys..

[41]  Mats Hamrud,et al.  A Partitioned Global Address Space implementation of the European Centre for Medium Range Weather Forecasts Integrated Forecasting System , 2015, Int. J. High Perform. Comput. Appl..

[42]  Almut Gassmann,et al.  A Consistent Time-Split Numerical Scheme Applied to the Nonhydrostatic Compressible Equations* , 2007 .

[43]  R. Lewis,et al.  Low-storage, Explicit Runge-Kutta Schemes for the Compressible Navier-Stokes Equations , 2000 .

[44]  Louis J. Wicker,et al.  A Time-Splitting Scheme for the Elastic Equations Incorporating Second-Order Runge–Kutta Time Differencing , 1998 .

[45]  Wojciech W. Grabowski,et al.  Towards Global Large Eddy Simulation: Super-Parameterization Revisited , 2016 .

[46]  W. Skamarock,et al.  The stability of time-split numerical methods for the hydrostatic and the nonhydrostatic elastic equations , 1992 .

[47]  Zdzisław Jackiewicz,et al.  General Linear Methods for Ordinary Differential Equations: Jackiewicz/General Linear , 2009 .

[48]  Pierre Bénard,et al.  Integration of the fully elastic equations cast in the hydrostatic pressure terrain-following coordinate in the framework of the ARPEGE/Aladin NWP system , 1995 .

[49]  G. Strang.  On the Construction and Comparison of Difference Schemes , 1968 .

[50]  Michail Diamantakis,et al.  Sensitivity of the ECMWF Model to Semi-Lagrangian Departure Point Iterations , 2016 .

[51]  J. Holton,et al.  Stratosphere‐troposphere exchange , 1995 .

[52]  Francis X. Giraldo,et al.  Lagrange—Galerkin methods on spherical geodesic grids: the shallow water equations , 2000 .

[53]  Todd D. Ringler,et al.  A Multiscale Nonhydrostatic Atmospheric Model Using Centroidal Voronoi Tesselations and C-Grid Staggering , 2012 .

[54]  Adam A. Scaife,et al.  The role of the stratosphere in the European climate response to El Niño , 2009 .

[55]  Markus Clemens,et al.  GPU Accelerated Adams–Bashforth Multirate Discontinuous Galerkin FEM Simulation of High-Frequency Electromagnetic Fields , 2010, IEEE Transactions on Magnetics.

[56]  William M. Putman,et al.  Cloud‐system resolving simulations with the NASA Goddard Earth Observing System global atmospheric model (GEOS‐5) , 2011 .

[57]  A. Staniforth,et al.  Semi-Lagrangian integration schemes for atmospheric models - A review , 1991 .

[58]  J. McGregor,et al.  The CSIRO Conformal-Cubic Atmospheric GCM , 2001 .

[59]  Shian‐Jiann Lin.  A “Vertically Lagrangian” Finite-Volume Dynamical Core for Global Models , 2004 .

[60]  Almut Gassmann,et al.  A global hexagonal C‐grid non‐hydrostatic dynamical core (ICON‐IAP) designed for energetic consistency , 2013 .

[61]  W. Schiesser.  The Numerical Method of Lines: Integration of Partial Differential Equations , 1991 .

[62]  Louis J. Wicker.  A Two-Step Adams-Bashforth-Moulton Split-Explicit Integrator for Compressible Atmospheric Models , 2009 .

[63]  Andrew Stuart,et al.  Earth System Modeling 2.0: A Blueprint for Models That Learn From Observations and Targeted High‐Resolution Simulations , 2017, 1709.00037.

[64]  D. Durran.  Numerical methods for wave equations in geophysical fluid dynamics , 1999 .

[65]  Steven J. Ruuth,et al.  Implicit-explicit methods for time-dependent partial differential equations , 1995 .

[66]  Nigel Wood,et al.  Runge-Kutta IMEX schemes for the Horizontally Explicit/Vertically Implicit (HEVI) solution of wave equations , 2013, J. Comput. Phys..

[67]  J. Klemp,et al.  The Simulation of Three-Dimensional Convective Storm Dynamics , 1978 .

[68]  Gianmarco Mengaldo,et al.  Discontinuous spectral/hp element methods: development, analysis and applications to compressible flows , 2015 .

[69]  A. Jameson,et al.  Numerical solution of the Euler equations by finite volume methods using Runge Kutta time stepping schemes , 1981 .

[70]  R. Heikes,et al.  DCMIP2016: A Review of Non-hydrostatic Dynamical Core Design and Intercomparison of Participating Models , 2017 .

[71]  Karim Yessad,et al.  The hydrostatic and nonhydrostatic global model IFS / ARPEGE : deep-layer model formulation and testing , 2012 .

[72]  P. Smolarkiewicz,et al.  A class of semi-Lagrangian approximations for fluids. , 1992 .

[73]  Peter Bauer,et al.  Atlas : A library for numerical weather prediction and climate modelling , 2017, Comput. Phys. Commun..

[74]  D. Keyes,et al.  Jacobian-free Newton-Krylov methods: a survey of approaches and applications , 2004 .

[75]  Nigel Wood,et al.  An inherently mass‐conserving iterative semi‐implicit semi‐Lagrangian discretization of the non‐hydrostatic vertical‐slice equations , 2010 .

[76]  Janusz A. Pudykiewicz,et al.  An efficient exponential time integration method for the numerical solution of the shallow water equations on the sphere , 2016, J. Comput. Phys..

[77]  Kazuo Saito,et al.  The Operational JMA Nonhydrostatic Mesoscale Model , 2006 .

[78]  Pierre Benard,et al.  Stability of Semi-Implicit and Iterative Centered-Implicit Time Discretizations for Various Equation Systems Used in NWP , 2003, physics/0304114.

[79]  Emil M. Constantinescu,et al.  Implicit-Explicit Formulations of a Three-Dimensional Nonhydrostatic Unified Model of the Atmosphere (NUMA) , 2013, SIAM J. Sci. Comput..

[80]  Mats Hamrud,et al.  A finite-volume module for simulating global all-scale atmospheric flows , 2016, J. Comput. Phys..

[81]  Robert Klöfkorn,et al.  Horizontally Explicit and Vertically Implicit (HEVI) Time Discretization Scheme for a Discontinuous Galerkin Nonhydrostatic Model , 2015 .

[82]  Andrew Staniforth,et al.  Aspects of the dynamical core of a nonhydrostatic, deep-atmosphere, unified weather and climate-prediction model , 2008, J. Comput. Phys..

[83]  J. Butcher.  The numerical analysis of ordinary differential equations: Runge-Kutta and general linear methods , 1987 .

[84]  William C. Skamarock,et al.  A time-split nonhydrostatic atmospheric model for weather research and forecasting applications , 2008, J. Comput. Phys..

[85]  Clive Temperton,et al.  An Efficient Two‐Time‐Level Semi‐Lagrangian Semi‐Implicit Integration Scheme , 1987 .

[86]  Torsten Hoefler,et al.  Near-global climate simulation at 1 km resolution: establishing a performance baseline on 4888 GPUs with COSMO 5.0 , 2017 .

[87]  Nigel Wood,et al.  GungHo! A new dynamical core for the Unified Model∗ , 2013 .

[88]  Mariano Hortal,et al.  The development and testing of a new two‐time‐level semi‐Lagrangian scheme (SETTLS) in the ECMWF forecast model , 2002 .

[89]  Emil M. Constantinescu,et al.  Acceleration of the IMplicit–EXplicit nonhydrostatic unified model of the atmosphere on manycore processors , 2017, Int. J. High Perform. Comput. Appl..

[90]  Takemasa Miyoshi,et al.  The Non-hydrostatic Icosahedral Atmospheric Model: description and development , 2014, Progress in Earth and Planetary Science.

[91]  Paul J. Kushner,et al.  Tropospheric response to stratospheric perturbations in a relatively simple general circulation model , 2002 .

[92]  Francis X. Giraldo,et al.  Semi-Implicit Formulations of the Navier--Stokes Equations: Application to Nonhydrostatic Atmospheric Modeling , 2010, SIAM J. Sci. Comput..

[93]  Nils Gustafsson,et al.  Four-dimensional ensemble variational (4D-En-Var) data assimilation for the HIgh Resolution Limited Area Model (HIRLAM) , 2014 .

[94]  Terry Davies,et al.  An iterative time‐stepping scheme for the Met Office's semi‐implicit semi‐Lagrangian non‐hydrostatic model , 2007 .

[95]  Nils Wedi,et al.  How does subgrid‐scale parametrization influence nonlinear spectral energy fluxes in global NWP models? , 2016 .

[96]  Michael Baldauf,et al.  Linear Stability Analysis of Runge–Kutta-Based Partial Time-Splitting Schemes for the Euler Equations , 2010 .

[97]  Michael Baldauf,et al.  Consortium for Small-Scale Modelling Technical Report No . 21 A new fast-waves solver for the Runge-Kutta dynamical core by Michael Baldauf April 2013 , 2013 .

[98]  Francis X. Giraldo,et al.  The Lagrange-Galerkin Spectral Element Method on Unstructured Quadrilateral Grids , 1998 .

[99]  Rolf Rannacher,et al.  Multiple Shooting and Time Domain Decomposition Methods , 2015 .

[100]  Pierre Bénard,et al.  RK‐IMEX HEVI schemes for fully compressible atmospheric models with advection: analyses and numerical testing , 2017 .

[101]  Chao Yang,et al.  10M-Core Scalable Fully-Implicit Solver for Nonhydrostatic Atmospheric Dynamics , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[102]  M. Diamantakis,et al.  An inherently mass‐conserving semi‐implicit semi‐Lagrangian discretization of the deep‐atmosphere global non‐hydrostatic equations , 2014 .

[103]  C. Kühnlein,et al.  The modelling infrastructure of the Integrated Forecasting System : Recent advances and future challenges , 2015 .

[104]  Giovanni Tumolo,et al.  A semi‐implicit, semi‐Lagrangian discontinuous Galerkin framework for adaptive numerical weather prediction , 2015 .

[105]  Christian Kühnlein,et al.  A consistent framework for discrete integrations of soundproof and compressible PDEs of atmospheric dynamics , 2014, J. Comput. Phys..

[106]  Christiane Jablonowski,et al.  Operator-Split Runge-Kutta-Rosenbrock Methods for Nonhydrostatic Atmospheric Models , 2012 .

[107]  D. Durran.  A physically motivated approach for filtering acoustic waves from the equations governing compressible stratified flow , 2008, Journal of Fluid Mechanics.

[108]  A. Staniforth,et al.  A new dynamical core for the Met Office's global and regional modelling of the atmosphere , 2005 .

[109]  Masaki Satoh,et al.  Nonhydrostatic icosahedral atmospheric model (NICAM) for global cloud resolving simulations , 2008, J. Comput. Phys..

[110]  Silvia Ferrari,et al.  A constrained integration (CINT) approach to solving partial differential equations using artificial neural networks , 2015, Neurocomputing.

[111]  Nigel Wood,et al.  Numerical analyses of Runge–Kutta implicit–explicit schemes for horizontally explicit, vertically implicit solutions of atmospheric models , 2014 .

[112]  Shian-Jiann Lin,et al.  A Two-Way Nested Global-Regional Dynamical Core on the Cubed-Sphere Grid , 2013 .

[113]  Louis J. Wicker,et al.  Time-Splitting Methods for Elastic Models Using Forward Time Schemes , 2002 .

[114]  M. Hochbruck,et al.  Exponential integrators , 2010, Acta Numerica.