Towards a Mini-App for Smoothed Particle Hydrodynamics at Exascale

The smoothed particle hydrodynamics (SPH) technique is a purely Lagrangian method, used in numerical simulations of fluids in astrophysics and computational fluid dynamics, among many other fields. SPH simulations with detailed physics represent computationally-demanding calculations. The parallelization of SPH codes is not trivial due to the absence of a structured grid. Additionally, the performance of the SPH codes can be, in general, adversely impacted by several factors, such as multiple time-stepping, long-range interactions, and/or boundary conditions. This work presents insights into the current performance and functionalities of three SPH codes: SPHYNX, ChaNGa, and SPH-flow. These codes are the starting point of an interdisciplinary co-design project, SPH-EXA, for the development of an Exascale-ready SPH mini-app. To gain such insights, a rotating square patch test was implemented as a common test simulation for the three SPH codes and analyzed on two modern HPC systems. Furthermore, to stress the differences with the codes stemming from the astrophysics community (SPHYNX and ChaNGa), an additional test case, the Evrard collapse, has also been carried out. This work extrapolates the common basic SPH features in the three codes for the purpose of consolidating them into a pure-SPH, Exascale-ready, optimized, mini-app. Moreover, the outcome of this serves as direct feedback to the parent codes, to improve their performance and overall scalability.

[1]  Bertrand Alessandrini,et al.  Smoothed Particle hydrodynamics simulation of fast ship flows , 2008 .

[2]  Yves Robert,et al.  Optimal Resilience Patterns to Cope with Fail-Stop and Silent Errors , 2016, 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[3]  Sandia Report,et al.  Improving Performance via Mini-applications , 2009 .

[4]  L. Lucy A numerical approach to the testing of the fission hypothesis. , 1977 .

[5]  Danilo Guerrera,et al.  Reproducible Stencil Compiler Benchmarks Using PROVA! , 2016, 2016 7th International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS).

[6]  Franck Cappello,et al.  Toward Exascale Resilience , 2009, Int. J. High Perform. Comput. Appl..

[7]  M. Rampp,et al.  Supernova Simulations with Boltzmann Neutrino Transport: A Comparison of Methods , 2003, astro-ph/0310662.

[8]  Bradford L. Chamberlain,et al.  Parallel Programmability and the Chapel Language , 2007, Int. J. High Perform. Comput. Appl..

[9]  James Reinders,et al.  Intel® threading building blocks , 2008 .

[10]  Peter Bauer,et al.  Atlas : A library for numerical weather prediction and climate modelling , 2017, Comput. Phys. Commun..

[11]  The CESAR Codesign Center: Early Results , 2012 .

[12]  Lukasz Wesolowski,et al.  Adaptive techniques for clustered N-body cosmological simulations , 2014, 1409.1929.

[13]  Albino Perego,et al.  Core-collapse supernovae in the hall of mirrors , 2018, Astronomy & Astrophysics.

[14]  Hartmut Kaiser,et al.  HPX: A Task Based Programming Model in a Global Address Space , 2014, PGAS.

[15]  J. Monaghan,et al.  Smoothed particle hydrodynamics: Theory and application to non-spherical stars , 1977 .

[16]  John M. Dennis,et al.  The CGPOP Miniapp , Version 1 . 0 , 2011 .

[17]  S. Rosswog,et al.  Boosting the accuracy of SPH techniques : Newtonian and special-relativistic tests , 2014, 1405.6034.

[18]  Joseph P. Kenny,et al.  Compiler-Assisted Source-to-Source Skeletonization of Application Models for System Simulation , 2018, ISC.

[19]  Switzerland,et al.  Improving smoothed particle hydrodynamics with an integral approach to calculating gradients , 2011, 1111.3261.

[20]  Ioana Banicescu,et al.  On the Scalability of Dynamic Scheduling Scientific Applications with Adaptive Weighted Factoring , 2003, Cluster Computing.

[21]  Ian Karlin,et al.  LULESH 2.0 Updates and Changes , 2013 .

[22]  L. Alvisi,et al.  A Survey of Rollback-Recovery Protocols , 2002 .

[23]  Osman S. Unsal,et al.  Unprotected Computing: A Large-Scale Study of DRAM Raw Error Rate on a Supercomputer , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[24]  Leslie Lamport,et al.  Distributed snapshots: determining global states of distributed systems , 1985, TOCS.

[25]  Franck Cappello,et al.  Toward Exascale Resilience: 2014 update , 2014, Supercomput. Front. Innov..

[26]  Piet Hut,et al.  A hierarchical O(N log N) force-calculation algorithm , 1986, Nature.

[27]  J. Figueira,et al.  SPHYNX: an accurate density-based SPH method for astrophysical applications , 2016, 1607.01698.

[28]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[29]  Peter Bauer,et al.  Energy-efficient SCalable Algorithms for weather Prediction at Exascale , 2017 .

[30]  David Le Touzé,et al.  On distributed memory MPI-based parallelization of SPH codes in massive HPC context , 2016, Comput. Phys. Commun..

[31]  Rubén M. Cabezón,et al.  A one-parameter family of interpolating kernels for smoothed particle hydrodynamics studies , 2008, J. Comput. Phys..

[32]  Anthony T. Chronopoulos,et al.  Dynamic multi phase scheduling for heterogeneous clusters , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[33]  Eduardo F. D'Azevedo,et al.  Developing MiniApps on Modern Platforms Using Multiple Programming Models , 2015, 2015 IEEE International Conference on Cluster Computing.

[34]  Daniel J. Price Smoothed particle hydrodynamics and magnetohydrodynamics , 2010, J. Comput. Phys..

[35]  John Shalf,et al.  Memory Errors in Modern Systems: The Good, The Bad, and The Ugly , 2015, ASPLOS.

[36]  Salvatore Marrone,et al.  SPH accuracy improvement through the combination of a quasi-Lagrangian shifting transport velocity and consistent ALE formalisms , 2016, J. Comput. Phys..

[37]  Franck Cappello,et al.  Addressing failures in exascale computing , 2014, Int. J. High Perform. Comput. Appl..

[38]  Ryan Marcus MCMini: Monte Carlo on GPGPU , 2012 .

[39]  Franck Cappello,et al.  Toward an Optimal Online Checkpoint Solution under a Two-Level HPC Checkpoint Model , 2017, IEEE Transactions on Parallel and Distributed Systems.

[40]  J. Monaghan,et al.  Fundamental differences between SPH and grid methods , 2006, astro-ph/0610051.

[41]  Arch D. Robison,et al.  Composable Parallel Patterns with Intel Cilk Plus , 2013, Computing in Science & Engineering.

[42]  W. Marsden I and J , 2012 .

[43]  E. Tasker,et al.  A test suite for quantitative comparison of hydrodynamic codes in astrophysics , 2008, 0808.1844.

[44]  A. Evrard Beyond N-body: 3D cosmological gas dynamics , 1988 .