A highly parallel multilevel Newton-Krylov-Schwarz method with subspace-based coarsening and partition-based balancing for the multigroup neutron transport equations on 3D unstructured meshes

The multigroup neutron transport equations have been widely used to study the motion of neutrons and their interactions with the background materials. Numerical simulation of the multigroup neutron transport equations is computationally challenging because the equations is defined on a high dimensional phase space (1D in energy, 2D in angle, and 3D in spatial space), and furthermore, for realistic applications, the computational spatial domain is complex and the materials are heterogeneous. The multilevel domain decomposition methods is one of the most popular algorithms for solving the multigroup neutron transport equations, but the construction of coarse spaces is expensive and often not strongly scalable when the number of processor cores is large. In this paper, we study a highly parallel multilevel Newton-Krylov-Schwarz method equipped with several novel components, such as subspace-based coarsening, partition-based balancing and hierarchical mesh partitioning, that enable the overall simulation strongly scalable in terms of the compute time. Compared with the traditional coarsening method, the subspace-based coarsening algorithm significantly reduces the cost of the preconditioner setup that is often unscalable. In addition, the partition-based balancing strategy enhances the parallel efficiency of the overall solver by assigning a nearly-equal amount of work to each processor core. The hierarchical mesh partitioning is able to generate a large number of subdomains and meanwhile minimizes the off-node communication. We numerically show that the proposed algorithm is scalable with more than 10,000 processor cores for a realistic application with a few billions unknowns on 3D unstructured meshes.

[1]  Robert D. Falgout,et al.  Coarse-Grid Selection for Parallel Algebraic Multigrid , 1998, IRREGULAR.

[2]  Thomas M. Evans,et al.  Multigrid in energy preconditioner for Krylov solvers , 2013, J. Comput. Phys..

[3]  Andrea Toselli,et al.  Domain decomposition methods : algorithms and theory , 2005 .

[4]  Xiao-Chuan Cai,et al.  A Highly Scalable Multilevel Schwarz Method with Boundary Geometry Preserving Coarse Spaces for 3D Elasticity Problems on Domains with Complex Geometry , 2016, SIAM J. Scientific Computing.

[5]  Michael Luby,et al.  A simple parallel algorithm for the maximal independent set problem , 1985, STOC '85.

[6]  David Andrs,et al.  A fully coupled two‐level Schwarz preconditioner based on smoothed aggregation for the transient multigroup neutron diffusion equations , 2018, Numer. Linear Algebra Appl..

[7]  J. W. Ruge,et al.  4. Algebraic Multigrid , 1987 .

[8]  Xiao-Chuan Cai,et al.  A scalable nonlinear fluid-structure interaction solver based on a Schwarz preconditioner with isogeometric unstructured coarse spaces in 3D , 2017, J. Comput. Phys..

[9]  Roy H. Stogner,et al.  A General-Purpose Hierarchical Mesh Partitioning Method with Node Balancing Strategies for Large-Scale Numerical Simulations , 2018, 2018 IEEE/ACM 9th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems (scalA).

[10]  Xiao-Chuan Cai,et al.  Simulation of unsteady blood flows in a patient‐specific compliant pulmonary artery with a highly parallel monolithically coupled fluid‐structure interaction algorithm , 2018, International journal for numerical methods in biomedical engineering.

[11]  Xiao-Chuan Cai,et al.  An efficient parallel simulation of unsteady blood flows in patient‐specific pulmonary artery , 2018, International journal for numerical methods in biomedical engineering.

[12]  Jim E. Morel,et al.  Angular Multigrid Preconditioner for Krylov-Based Solution Techniques Applied to the Sn Equations with Highly Forward-Peaked Scattering , 2012 .

[13]  Andrew R. Siegel,et al.  Enabling high-fidelity neutron transport simulations on petascale architectures , 2009, Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis.

[14]  Vincent Laboure,et al.  Rattlesnake Theory Manual , 2018 .

[15]  George Karypis,et al.  Parmetis parallel graph partitioning and sparse matrix ordering library , 1997 .

[16]  Fande Kong,et al.  A Parallel Implicit Fluid-structure Interaction Solver with Isogeometric Coarse Spaces for 3D Unstructured Mesh Problems with Complex Geometry , 2016 .

[17]  T. Hughes,et al.  Streamline upwind/Petrov-Galerkin formulations for convection dominated flows with particular emphasis on the incompressible Navier-Stokes equations , 1990 .

[18]  Jean-Jacques Lautard,et al.  DOMAIN DECOMPOSITION METHODS FOR CORE CALCULATIONS USING THE MINOS SOLVER , 2007 .

[19]  Ulrike Meier Yang,et al.  Parallel Algebraic Multigrid Methods — High Performance Preconditioners , 2006 .

[20]  Xiao-Chuan Cai,et al.  Scalability study of an implicit solver for coupled fluid-structure interaction problems on unstructured meshes in 3D , 2018, Int. J. High Perform. Comput. Appl..

[21]  E. Lewis,et al.  Computational Methods of Neutron Transport , 1993 .

[22]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[23]  S. Van Criekingen,et al.  parafish: A parallel FE–PN neutron transport solver based on domain decomposition , 2011 .

[24]  Richard C. Martineau,et al.  A flexible nonlinear diffusion acceleration method for the SN transport equations discretized with discontinuous finite elements , 2017, J. Comput. Phys..

[25]  Barry F. Smith,et al.  Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations , 1996 .

[26]  François Pellegrini,et al.  PT-Scotch: A tool for efficient parallel graph ordering , 2008, Parallel Comput..

[27]  R. E. Alcouffe,et al.  Diffusion synthetic acceleration methods for the diamond-differenced discrete-ordinates equations , 1977 .

[28]  V. E. Henson,et al.  BoomerAMG: a parallel algebraic multigrid solver and preconditioner , 2002 .

[29]  U. Yang,et al.  Distance-two interpolation for parallel algebraic multigrid , 2007 .

[30]  D. Keyes,et al.  Jacobian-free Newton-Krylov methods: a survey of approaches and applications , 2004 .

[31]  Zeyao Mo,et al.  Parallel Flux Sweep Algorithm for Neutron Transport on Unstructured Grid , 2004, The Journal of Supercomputing.

[32]  B. CHANG,et al.  Spatial Multigrid for Isotropic Neutron Transport , 2007, SIAM J. Sci. Comput..

[33]  K. Stuben,et al.  Algebraic Multigrid (AMG) : An Introduction With Applications , 2000 .