Optimisation of Patch Distribution Strategies for AMR Applications

As core counts increase in the world's most powerful supercomputers, applications are becoming limited not only by computational power, but also by data availability. In the race to exascale, efficient and effective communication policies are key to achieving optimal application performance. Applications using adaptive mesh refinement (AMR) trade off communication for computational load balancing, to enable the focused computation of specific areas of interest. This class of application is particularly susceptible to the communication performance of the underlying architectures, and are inherently difficult to scale efficiently. In this paper we present a study of the effect of patch distribution strategies on the scalability of an AMR code. We demonstrate the significance of patch placement on communication overheads, and by balancing the computation and communication costs of patches, we develop a scheme to optimise performance of a specific, industry-strength, benchmark application.

[1]  M. Berger,et al.  Adaptive mesh refinement for hyperbolic partial differential equations , 1982 .

[2]  Zhiling Lan,et al.  Dynamic load balancing for structured adaptive mesh refinement applications , 2001, International Conference on Parallel Processing, 2001..

[3]  Mary K. Vernon,et al.  A plug-and-play model for evaluating wavefront computations on parallel architectures , 2008, 2008 IEEE International Symposium on Parallel and Distributed Processing.

[4]  J. Quirk.  A parallel adaptive grid algorithm for computational shock hydrodynamics , 1996 .

[5]  Jack Dongarra,et al.  Recent Advances in Parallel Virtual Machine and Message Passing Interface, 15th European PVM/MPI Users' Group Meeting, Dublin, Ireland, September 7-10, 2008. Proceedings , 2008, PVM/MPI.

[6]  Antonio Baeza,et al.  Adaptive mesh refinement techniques for high‐order shock capturing schemes for multi‐dimensional hydrodynamic simulations , 2006 .

[7]  B. Fryxell,et al.  FLASH: An Adaptive Mesh Hydrodynamics Code for Modeling Astrophysical Thermonuclear Flashes , 2000 .

[8]  Shweta Sinha,et al.  Adaptive runtime partitioning of AMR applications on heterogeneous clusters , 2001, Proceedings 42nd IEEE Symposium on Foundations of Computer Science.

[9]  John B. Bell,et al.  Performance and scaling of locally-structured grid methods forpartial differential equations , 2007 .

[10]  John Shalf,et al.  Scalability challenges for massively parallel AMR applications , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[11]  Jingjin Wu,et al.  Performance Emulation of Cell-Based AMR Cosmology Simulations , 2011, 2011 IEEE International Conference on Cluster Computing.

[12]  Greg L. Bryan,et al.  Fluids in the universe: adaptive mesh refinement in cosmology , 1999, Comput. Sci. Eng..

[13]  A.M. Wissink,et al.  Large Scale Parallel Structured AMR Calculations Using the SAMRAI Framework , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[14]  Mark F. Adams,et al.  Chombo Software Package for AMR Applications Design Document , 2014 .

[15]  Simon D. Hammond,et al.  Benchmarking and modelling of POWER7, Westmere, BG/P, and GPUs: an industry case study , 2011, PERV.

[16]  Isidore Rigoutsos,et al.  An algorithm for point clustering and grid generation , 1991, IEEE Trans. Syst. Man Cybern..

[17]  M. Aurada,et al.  Convergence of adaptive BEM for some mixed boundary value problem , 2012, Applied numerical mathematics : transactions of IMACS.

[18]  P. Colella,et al.  Local adaptive mesh refinement for shock hydrodynamics , 1989 .

[19]  Ralf H. Reussner,et al.  SKaMPI: A Detailed, Accurate MPI Benchmark , 1998, PVM/MPI.

[20]  Stephen A. Jarvis,et al.  Predictive analysis of a hydrodynamics application on large-scale CMP clusters , 2011, Computer Science - Research and Development.

[21]  Justin Luitjens,et al.  Scalable parallel regridding algorithms for block‐structured adaptive mesh refinement , 2011, Concurr. Comput. Pract. Exp..