A stencil-based implementation of Parareal in the C++ domain specific embedded language STELLA

In view of the rapid rise of the number of cores in modern supercomputers, time-parallel methods that introduce concurrency along the temporal axis are becoming increasingly popular. For the solution of time-dependent partial differential equations, these methods can add another direction for concurrency on top of spatial parallelization. The paper presents an implementation of the time-parallel Parareal method in a C++ domain specific language for stencil computations (STELLA). STELLA provides both an OpenMP and a CUDA backend for a shared memory parallelization, using the CPU or GPU inside a node for the spatial stencils. Here, we intertwine this node-wise spatial parallelism with the time-parallel Parareal. This is done by adding an MPI-based implementation of Parareal, which allows us to parallelize in time across nodes. The performance of Parareal with both backends is analyzed in terms of speedup, parallel efficiency and energy-to-solution for an advection-diffusion problem with a time-dependent diffusion coefficient.

[1]  Charbel Farhat,et al.  Time‐parallel implicit integrators for the near‐real‐time prediction of linear structural dynamic responses , 2006 .

[2]  Rolf Krause,et al.  Explicit Parallel-in-time Integration of a Linear Acoustic-Advection System , 2012, ArXiv.

[3]  Y. Maday,et al.  A “Parareal” Time Discretization for Non-Linear PDE’s with Application to the Pricing of an American Put , 2002 .

[4]  Dominik Obrist,et al.  A Parallel Solver for the Time-Periodic Navier-Stokes Equations , 2013, PPAM.

[5]  Wu-chun Feng,et al.  The Green500 List: Encouraging Sustainable Supercomputing , 2007, Computer.

[6]  Rolf Krause,et al.  Parareal for Diffusion Problems with Space- and Time-Dependent Coefficients , 2014, 1401.7829.

[7]  Rolf Krause,et al.  A space-time parallel solver for the three-dimensional heat equation , 2013, PARCO.

[8]  Tobias Gysi,et al.  Towards a performance portable, architecture agnostic implementation strategy for weather and climate models , 2014, Supercomput. Front. Innov..

[9]  Dominik Obrist,et al.  A Parallel Space-Time Finite Difference Solver for Periodic Solutions of the Shallow-Water Equation , 2011, PPAM.

[10]  Raúl Sánchez,et al.  Mechanisms for the convergence of time-parallelized, parareal turbulent plasma simulations , 2012, J. Comput. Phys..

[11]  Einar M. Rønquist,et al.  Stability of the Parareal Algorithm , 2005 .

[12]  Robert D. Falgout,et al.  A Multigrid-in-Time Algorithm for Solving Evolution Equations in Parallel , 2012 .

[13]  Terry Haut,et al.  An Asymptotic Parallel-in-Time Method for Highly Oscillatory PDEs , 2013, SIAM J. Sci. Comput..

[14]  Jürg Nievergelt,et al.  Parallel methods for integrating ordinary differential equations , 1964, CACM.

[15]  W. Miranker,et al.  Parallel methods for the numerical integration of ordinary differential equations , 1967 .

[16]  Benjamin W. Ong,et al.  MPI–OpenMP Algorithms for the Parallel Space–Time Solution of Time Dependent PDEs , 2014 .

[17]  Gilles Fourestey,et al.  First Experiences With Validating and Using the Cray Power Management Database Tool , 2014, ArXiv.

[18]  M. Gander,et al.  Analysis of a Time Multigrid Algorithm for DG-Discretizations in Time , 2014, 1409.5254.

[19]  M. Kiehl,et al.  Parallel Multiple Shooting for the Solution of Initial Value Problems , 1994, Parallel Comput..

[20]  Graham Horton,et al.  The time‐parallel multigrid method , 1992 .

[21]  Martin J. Gander,et al.  Analysis of a Krylov subspace enhanced parareal algorithm for linear problems , 2008 .

[22]  J. Lions,et al.  Résolution d'EDP par un schéma en temps « pararéel » , 2001 .

[23]  James Demmel,et al.  the Parallel Computing Landscape , 2022 .

[24]  David E. Keyes,et al.  Exaflop/s: The why and the how , 2011 .

[25]  Charbel Farhat,et al.  Time‐decomposed parallel time‐integrators: theory and feasibility studies for fluid, structure, and fluid–structure applications , 2003 .

[26]  Xiaoying Dai,et al.  Stable Parareal in Time Method for First- and Second-Order Hyperbolic Systems , 2012, SIAM J. Sci. Comput..

[27]  Daniel Ruprecht,et al.  Convergence of Parareal with spatial coarsening , 2014 .

[28]  Andrew T. Barker,et al.  A minimal communication approach to parallel time integration , 2013, Int. J. Comput. Math..

[29]  Martin J. Gander,et al.  Domain Decomposition Methods in Science and Engineering XXI , 2014 .

[30]  Andrew J. Christlieb,et al.  A Parallel Space-Time Algorithm , 2012, SIAM J. Sci. Comput..

[31]  Colin B. Macdonald,et al.  Parallel High-Order Integrators , 2010, SIAM J. Sci. Comput..

[32]  Martin J. Gander,et al.  50 Years of Time Parallel Time Integration , 2015 .

[33]  Rolf Krause,et al.  A massively space-time parallel N-body solver , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[34]  Hans Werner Meuer,et al.  Top500 Supercomputer Sites , 1997 .

[35]  Michael L. Minion,et al.  TOWARD AN EFFICIENT PARALLEL IN TIME METHOD FOR PARTIAL DIFFERENTIAL EQUATIONS , 2012 .

[36]  Michael L. Minion,et al.  A HYBRID PARAREAL SPECTRAL DEFERRED CORRECTIONS METHOD , 2010 .

[37]  Graham Horton,et al.  An Algorithm with Polylog Parallel Complexity for Solving Parabolic Partial Differential Equations , 1995, SIAM J. Sci. Comput..

[38]  David I. Ketcheson,et al.  Communications in Applied Mathematics and Computational Science , 2012 .

[39]  Wolfgang Hackbusch,et al.  Parabolic multi-grid methods , 1985 .

[40]  Martin J. Gander,et al.  PARAEXP: A Parallel Integrator for Linear Initial-Value Problems , 2013, SIAM J. Sci. Comput..

[41]  Martin J. Gander,et al.  Analysis of the Parareal Time-Parallel Time-Integration Method , 2007, SIAM J. Sci. Comput..

[42]  Tobias Gysi,et al.  STELLA: A domain-specific embedded language for stencil codes on structured grids , 2014 .

[43]  Jose C. F. Pereira,et al.  Parallel-in-Time Simulation of Two-Dimensional, Unsteady, Incompressible Laminar Flows , 2006 .

[44]  Guillaume Bal,et al.  On the Convergence and the Stability of the Parareal Algorithm to Solve Partial Differential Equations , 2005 .

[45]  Yvon Maday,et al.  A Parareal in Time Semi-implicit Approximation of the Navier-Stokes Equations , 2005 .

[46]  Raúl Sánchez,et al.  Event-based parareal: A data-flow based implementation of parareal , 2012, J. Comput. Phys..

[47]  Raúl Sánchez,et al.  Parallelization in time of numerical simulations of fully-developed plasma turbulence using the parareal algorithm , 2010, J. Comput. Phys..

[48]  Gilles Fourestey,et al.  Application Centric Energy-Efficiency Study of Distributed Multi-Core and Hybrid CPU-GPU Systems , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[49]  Yvon Maday,et al.  Parallel in time algorithms for quantum control: Parareal time discretization scheme , 2003 .

[50]  Martin J. Gander,et al.  Analysis for parareal algorithms applied to Hamiltonian differential equations , 2014, J. Comput. Appl. Math..