The Plasma Simulation Code: A modern particle-in-cell code with load-balancing and GPU support

Recent increases in supercomputing power, driven by the multi-core revolution and accelerators such as the IBM Cell processor, graphics processing units (GPUs) and Intel's Many Integrated Core (MIC) technology have enabled kinetic simulations of plasmas at unprecedented resolutions, but changing HPC architectures also come with challenges for writing efficient numerical codes. This paper describes the Plasma Simulation Code (PSC), an explicit, electromagnetic particle-in-cell code with support for different order particle shape functions. We focus on two distinguishing feature of the code: patch-based load balancing using space-filling curves, and support for Nvidia GPUs, which achieves substantial speed-up of up to more than 6x on the Cray XK7 architecture compared to a CPU-only implementation.

[1]  Wei Lu,et al.  OSIRIS: A Three-Dimensional, Fully Relativistic Particle in Cell Code for Modeling Plasma Based Accelerators , 2002, International Conference on Computational Science.

[2]  Alexander Pukhov,et al.  Three-dimensional electromagnetic relativistic particle-in-cell code VLPL (Virtual Laser Plasma Lab) , 1999, Journal of Plasma Physics.

[3]  P. Chang,et al.  Filamentation instability of counterstreaming laser-driven plasmas. , 2013, Physical review letters.

[4]  Kai Germaschewski,et al.  Using Automated Code Generation to Support High Performance Extended MHD Integration in OpenGGCM , 2011 .

[5]  R W Hockney,et al.  Computer Simulation Using Particles , 1966 .

[6]  Nail A. Gumerov,et al.  Fast parallel Particle-To-Grid interpolation for plasma PIC simulations on the GPU , 2008, J. Parallel Distributed Comput..

[7]  L. Greengard,et al.  Regular Article: A Fast Adaptive Multipole Algorithm in Three Dimensions , 1999 .

[8]  Kai Germaschewski,et al.  The Magnetic Reconnection Code: an AMR-based fully implicit simulation suite , 2009 .

[9]  William Gropp,et al.  Efficient Management of Parallelism in Object-Oriented Numerical Software Libraries , 1997, SciTools.

[10]  Kai Germaschewski,et al.  New Developments in Modeling MHD Systems on High Performance Computing Architectures , 2009 .

[11]  William Daughton,et al.  Fully kinetic simulations of undriven magnetic reconnection with open boundary conditions , 2006 .

[12]  V. G. Weirs,et al.  On Validating an Astrophysical Simulation Code , 2002, astro-ph/0206251.

[13]  Liang Wang,et al.  Comparison of multi-fluid moment models with Particle-in-Cell simulations of collisionless magnetic reconnection , 2014, 1409.0262.

[14]  Viktor K. Decyk,et al.  Particle-in-Cell algorithms for emerging computer architectures , 2014, Comput. Phys. Commun..

[15]  A. Stamm,et al.  Variational formulation of particle algorithms for kinetic electromagnetic plasma simulations , 2013, 2013 Abstracts IEEE International Conference on Plasma Science (ICOPS).

[16]  G. Peano.  Sur une courbe, qui remplit toute une aire plane , 1890 .

[17]  Rony Keppens,et al.  A multidimensional grid-adaptive relativistic magnetofluid code , 2008, Comput. Phys. Commun..

[18]  Viktor K. Decyk,et al.  Dynamic load balancing for a 2D concurrent plasma PIC code , 1993 .

[19]  C. Birdsall,et al.  Plasma Physics via Computer Simulation , 2018 .

[20]  Giovanni Lapenta,et al.  Particle simulations of space weather , 2012, J. Comput. Phys..

[21]  K. Germaschewski,et al.  Fast magnetic reconnection in laser-produced plasma bubbles. , 2011, Physical review letters.

[22]  John D. Villasenor,et al.  Rigorous charge conservation for local electromagnetic field solvers , 1992 .

[23]  Kai Germaschewski,et al.  Magnetic reconnection in high-energy-density laser-produced plasmasa) , 2012 .

[24]  K. Bowers,et al.  Ultrahigh performance three-dimensional electromagnetic relativistic kinetic plasma simulationa) , 2008 .

[25]  H. Karimabadi,et al.  Coherent structures, intermittent turbulence, and dissipation in high-temperature plasmas , 2013 .

[26]  Igor V. Sokolov,et al.  Alternating-order interpolation in a charge-conserving scheme for particle-in-cell simulations , 2011, Comput. Phys. Commun..

[27]  P. Chang,et al.  Magnetic reconnection between colliding magnetized laser-produced plasma plumes. , 2014, Physical review letters.

[28]  A. Grimshaw,et al.  High Performance and Scalable Radix Sorting: a Case Study of Implementing Dynamic Parallelism for GPU Computing , 2011, Parallel Process. Lett..

[29]  David E. Keyes,et al.  Using Krylov-Schwarz methods in an adaptive mesh refinement environment , 2005 .

[30]  J. Raeder.  Global Magnetohydrodynamics — A Tutorial , 2003 .

[31]  William Daughton,et al.  A review of pressure anisotropy caused by electron trapping in collisionless plasma, and its implications for magnetic reconnection , 2013 .

[32]  Ümit V. Çatalyürek,et al.  Load-balancing spatially located computations using rectangular partitions , 2011, J. Parallel Distributed Comput..

[33]  Michael C. Huang,et al.  Particle-in-cell simulations with charge-conserving current deposition on graphic processing units , 2010, J. Comput. Phys..

[34]  K. Yee.  Numerical solution of initial boundary value problems involving maxwell's equations in isotropic media , 1966 .

[35]  J. Cary,et al.  VORPAL: a versatile plasma simulation code , 2004 .

[36]  William Daughton,et al.  Large-scale electron acceleration by parallel electric fields during magnetic reconnection , 2011, Nature Physics.

[37]  D. Hilbert.  Ueber die stetige Abbildung einer Line auf ein Flächenstück , 1891 .

[38]  Leslie Greengard,et al.  A fast algorithm for particle simulations , 1987 .

[39]  Michael Garland,et al.  Designing efficient sorting algorithms for manycore GPUs , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[40]  T. Esirkepov,et al.  Exact charge conservation scheme for Particle-in-Cell simulation with an arbitrary form-factor , 2001 .

[41]  Viktor K. Decyk,et al.  Adaptable Particle-in-Cell algorithms for graphical processing units , 2010, Comput. Phys. Commun..

[42]  H Burau,et al.  PIConGPU: A Fully Relativistic Particle-in-Cell Code for a GPU Cluster , 2010, IEEE Transactions on Plasma Science.

[43]  D. C. Barnes,et al.  A charge- and energy-conserving implicit, electrostatic particle-in-cell algorithm on mapped computational meshes , 2013, J. Comput. Phys..

[44]  A. Bruce Langdon,et al.  On enforcing Gauss' law in electromagnetic particle-in-cell codes , 1992 .

[45]  Kai Germaschewski,et al.  Current singularities: Drivers of impulsive reconnectiona) , 2005 .