Mixed-precision for Linear Solvers in Global Geophysical Flows

Semi-implicit time-stepping schemes for atmosphere and ocean models require elliptic solvers that work efficiently on modern supercomputers. This paper reports our study of the potential computational savings when using mixed precision arithmetic in the elliptic solvers. The essential components of a representative elliptic solver are run at precision levels as low as half (16 bits), and accompanied with a detailed evaluation of the impact of reduced precision on the solver convergence and the solution quality. A detailed inquiry into reduced precision requires a model configuration that is meaningful but cheaper to run and easier to evaluate than full atmosphere/ocean models. This study is therefore conducted in the context of a novel semi-implicit shallow-water model on the sphere, purposely designed to mimic numerical intricacies of modern all-scale weather and climate (W&C) models with the numerical stability independent on celerity of all wave motions. The governing algorithm of the shallow-water model is based on the non-oscillatory MPDATA methods for geophysical flows, whereas the resulting elliptic problem employs a strongly preconditioned non-symmetric Krylov-subspace solver GCR, proven in advanced atmospheric applications. The classical longitude/latitude grid is deliberately chosen to retain the stiffness of global W&C models posed in thin spherical shells as well as to better understand the performance of reduced-precision arithmetic in the vicinity of grid singularities. Precision reduction is done on a software level, using an emulator. The reduced-precision experiments are conducted for established dynamical-core test-cases, like the Rossby-Haurwitz wave number 4 and a zonal orographic flow. The study shows that selected key components of the elliptic solver, most prominently the preconditioning, can be performed at the level of half precision.

[1]  Peter D. Düben,et al.  Single Precision in Weather Forecasting Models: An Evaluation with the IFS , 2017 .

[2]  J. Szmelter,et al.  MPDATA: An edge-based unstructured-grid formulation , 2005 .

[3]  Peter Korn,et al.  Formulation of an unstructured grid model for global ocean dynamics , 2017, J. Comput. Phys..

[4]  C. M. Maynard,et al.  Mixed-precision arithmetic in the ENDGame dynamical core of the Unified Model, a numerical weather prediction and climate model code , 2019, Comput. Phys. Commun..

[5]  Len G. Margolin,et al.  On Forward-in-Time Differencing for Fluids: Stopping Criteria for Iterative Solutions of Anelastic Pressure Equations , 1997 .

[6]  J. Szmelter,et al.  A nonhydrostatic unstructured-mesh soundproof model for simulation of internal gravity waves , 2011 .

[7]  P. K. Smolarkiewicz,et al.  VARIATIONAL METHODS FOR ELLIPTIC PROBLEMS IN FLUID MODELS , 2000 .

[8]  Peter D. Düben,et al.  Posits as an alternative to floats for weather and climate models , 2019, Proceedings of the Conference for Next Generation Arithmetic 2019.

[9]  Hirofumi Tomita,et al.  Single Precision in the Dynamical Core of a Nonhydrostatic Global Atmospheric Model: Evaluation Using a Baroclinic Wave Test Case , 2018 .

[10]  Christian Kühnlein,et al.  An unstructured-mesh finite-volume MPDATA for compressible atmospheric dynamics , 2017, J. Comput. Phys..

[11]  Brian J. Hoskins,et al.  On the Influence of Orography on Large-Scale Atmospheric Flow. , 1979 .

[12]  Joanna Szmelter,et al.  An edge-based unstructured mesh discretisation in geospherical framework , 2010, J. Comput. Phys..

[13]  Brian J. Hoskins,et al.  Stability of the Rossby-Haurwitz wave , 1973 .

[14]  J. Prusa,et al.  EULAG, a computational model for multiscale flows , 2008 .

[15]  Patrick Heimbach,et al.  OVERVIEW OF THE FORMULATION AND NUMERICS OF THE MIT GCM . vnorth uwest ueast vsouth wup , 2004 .

[16]  Qiang Ye,et al.  Residual Replacement Strategies for Krylov Subspace Iterative Methods for the Convergence of True Residuals , 2000, SIAM J. Sci. Comput..

[17]  Luis Chacón,et al.  An efficient mixed-precision, hybrid CPU-GPU implementation of a nonlinearly implicit one-dimensional particle-in-cell algorithm , 2011, J. Comput. Phys..

[18]  Jens Schröter,et al.  The Finite Element Sea Ice-Ocean Model (FESOM) v.1.4: formulation of an ocean general circulation model , 2014 .

[19]  S. Eisenstat,et al.  Variational Iterative Methods for Nonsymmetric Systems of Linear Equations , 1983 .

[20]  Jack J. Dongarra,et al.  The Design of Fast and Energy-Efficient Linear Solvers: On the Potential of Half-Precision Arithmetic and Iterative Refinement Techniques , 2018, ICCS.

[21]  Peter D. Düben,et al.  rpe v5: an emulator for reduced floating-point precision in large numerical simulations , 2016 .

[22]  Joanna Szmelter,et al.  FVM 1.0: a nonhydrostatic finite-volume dynamical core for the IFS , 2019, Geoscientific Model Development.

[23]  Nicholas J. Higham,et al.  Accelerating the Solution of Linear Systems by Iterative Refinement in Three Precisions , 2018, SIAM J. Sci. Comput..

[24]  Christian Kühnlein,et al.  Semi-implicit integrations of perturbation equations for all-scale atmospheric dynamics , 2019, J. Comput. Phys..

[25]  Enrique S. Quintana-Ortí,et al.  Toward a modular precision ecosystem for high-performance computing , 2019, Int. J. High Perform. Comput. Appl..

[26]  Mats Hamrud,et al.  A finite-volume module for simulating global all-scale atmospheric flows , 2016, J. Comput. Phys..

[27]  Roger Temam,et al.  Suitable initial conditions , 2006, J. Comput. Phys..

[28]  Peter D. Düben,et al.  Improving Weather Forecast Skill through Reduced-Precision Data Assimilation , 2018 .

[29]  Christian Kühnlein,et al.  A finite-volume module for cloud-resolving simulations of global atmospheric flows , 2017, J. Comput. Phys..

[31]  Robert Strzodka,et al.  Performance and accuracy of hardware-oriented native-, emulated- and mixed-precision solvers in FEM simulations , 2007, Int. J. Parallel Emergent Distributed Syst..

[32]  Peter Messmer,et al.  The ESCAPE project: Energy-efficient Scalable Algorithms for Weather Prediction at Exascale , 2017, Geoscientific Model Development.

[33]  L. Margolin,et al.  MPDATA: A Finite-Difference Solver for Geophysical Flows , 1998 .

[34]  L. Margolin,et al.  A Class of Nonhydrostatic Global Models. , 2001 .

[35]  Peter D. Düben,et al.  Benchmark Tests for Numerical Weather Forecasts on Inexact Hardware , 2014 .

[36]  B. Haurwitz.  THE MOTION OF ATMOSPHERIC DISTURBANCES ON THE SPHERICAL EARTH , 2019 .

[37]  Yong Li,et al.  Numerical simulations of Rossby–Haurwitz waves , 2000 .

[38]  Peter D. Düben,et al.  The use of imprecise processing to improve accuracy in weather & climate prediction , 2014, J. Comput. Phys..

[39]  Francis X. Giraldo,et al.  Current and Emerging Time-Integration Strategies in Global Numerical Weather and Climate Prediction , 2019 .

[40]  Mikito Furuichi,et al.  Development of a Stokes flow solver robust to large viscosity jumps using a Schur complement approach with mixed precision arithmetic , 2011, J. Comput. Phys..

[41]  M. Ziemianski,et al.  Convection-permitting regional weather modeling with COSMO-EULAG: compressible and anelastic solutions for a typical westerly flow over the Alps , 2016 .

[42]  Joseph M. Prusa.  Computation at a coordinate singularity , 2018, J. Comput. Phys..

[43]  Julien Langou,et al.  Accelerating scientific computations with mixed precision algorithms , 2008, Comput. Phys. Commun..

[44]  Matthew Chantry,et al.  Scale-Selective Precision for Weather and Climate Forecasting , 2019, Monthly Weather Review.

[45]  P. Swarztrauber,et al.  A standard test set for numerical approximations to the shallow water equations in spherical geometry , 1992 .

[46]  Ana Cortés,et al.  How to use mixed precision in ocean models: exploring a potential reduction of numerical precision in NEMO 4.0 and ROMS 3.6 , 2019, Geoscientific Model Development.

[47]  Chao Yang,et al.  10M-Core Scalable Fully-Implicit Solver for Nonhydrostatic Atmospheric Dynamics , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[48]  Tim Palmer.  Towards the probabilistic Earth‐system simulator: a vision for the future of climate and weather prediction , 2012 .

[49]  T. Palmer,et al.  Reduced‐precision parametrization: lessons from an intermediate‐complexity atmospheric model , 2020, Quarterly Journal of the Royal Meteorological Society.

[50]  Danesh K. Tafti,et al.  Computational Fluid Dynamics Computations Using a Preconditioned Krylov Solver on Graphical Processing Units , 2016 .