Dynamic Load Balancing of Plasma and Flow Simulations

Extracting performance from simulations with complex information dependencies on massively parallel computers requires the computational work to be evenly distributed across the processing resources while maintaining low communication costs. Plasma simulations using a particle-in-cell method and computational fluid dynamics using unstructured mesh-based finite element and volume methods present three distinct distribution requirements. To meet these needs, we present EnGPar's diffusive partition improvement method. An initial demonstration of EnGPar's particle distribution improvement is provided along with fluid dynamics mesh partition improvement results on up to 512Ki processes on an IBM BlueGene/Q.

[1]  Edward A. Carmona,et al.  On Parallel PIC Versatility and the Structure of Parallel PIC Approaches , 1997, Concurr. Pract. Exp..

[2]  George Cybenko,et al.  Dynamic Load Balancing for Distributed Memory Multiprocessors , 1989, J. Parallel Distributed Comput..

[3]  George Karypis,et al.  Multi-threaded Graph Partitioning , 2013, 2013 IEEE 27th International Symposium on Parallel and Distributed Processing.

[4]  P. Wesseling,et al.  Geometric multigrid with applications to computational fluid dynamics , 2001 .

[5]  Vipin Kumar,et al.  Parallel Multilevel series k-Way Partitioning Scheme for Irregular Graphs , 1999, SIAM Rev..

[6]  Davide Curreli,et al.  hPIC: A scalable electrostatic Particle-in-Cell for Plasma-Material Interactions , 2018, Comput. Phys. Commun..

[7]  Onkar Sahni,et al.  Adaptive boundary layer meshing for viscous flow simulations , 2008, Engineering with Computers.

[8]  Steven J. Plimpton,et al.  A load-balancing algorithm for a parallel electromagnetic particle-in-cell code , 2003 .

[9]  Onkar Sahni,et al.  Anisotropic Adaptation for Transonic Flows with Turbulent Boundary Layers , 2015 .

[10]  David Green,et al.  GITR Simulation of Helium Exposed Tungsten Erosion and Redistribution in PISCES-A , 2017 .

[11]  Michael Gschwind,et al.  The IBM Blue Gene/Q Compute Chip , 2012, IEEE Micro.

[12]  Onkar Sahni,et al.  Unstructured mesh partition improvement for implicit finite element at extreme scale , 2010, The Journal of Supercomputing.

[13]  Mehmet Deveci,et al.  UMPa: A multi-objective, multi-level partitioner for communication minimization , 2012, Graph Partitioning and Graph Clustering.

[14]  Mark S. Shephard,et al.  Improving Unstructured Mesh Partitions for Multiple Criteria Using Mesh Adjacencies , 2018, SIAM J. Sci. Comput..

[15]  Choong-Seock Chang,et al.  Numerical study of neoclassical plasma pedestal in a tokamak geometry , 2004 .

[16]  Mark S. Shephard,et al.  Dynamic load balancing of massively parallel unstructured meshes , 2017, ScalA@SC.

[17]  Joshua Breslau,et al.  Multiple timescale calculations of sawteeth and other global macroscopic dynamics of tokamak plasmas , 2012 .

[18]  Vipin Kumar,et al.  Parallel static and dynamic multi‐constraint graph partitioning , 2002, Concurr. Comput. Pract. Exp..

[19]  Isaac D. Scherson,et al.  An analysis of diffusive load-balancing , 1994, SPAA '94.

[20]  Robert Hager,et al.  A new hybrid-Lagrangian numerical scheme for gyrokinetic simulation of tokamak edge plasma , 2016, J. Comput. Phys..

[21]  Eduardo F. D'Azevedo,et al.  Balancing Particle and Mesh Computation in a Particle-In-Cell Code , 2016 .

[22]  Choong-Seock Chang,et al.  Full-f gyrokinetic particle simulation of centrally heated global ITG turbulence from magnetic axis to edge pedestal top in a realistic tokamak geometry , 2009 .

[23]  Shahid H. Bokhari,et al.  A Partitioning Strategy for Nonuniform Problems on Multiprocessors , 1987, IEEE Transactions on Computers.

[24]  Mehmet Deveci,et al.  Multi-Jagged: A Scalable Parallel Spatial Partitioning Algorithm , 2016, IEEE Transactions on Parallel and Distributed Systems.

[25]  Adrien Loseille,et al.  On 3D Anisotropic Local Remeshing for Surface, Volume and Boundary Layers , 2009, IMR.