Parallelization of an Additive Multigrid Solver

This article deals with the implementation and performance analysis of a parallel algebraic multigrid solver (pAMG) for a finite-volume, unstructured computational fluid dynamics (CFD) code. The parallelization of the solver is based on the domain decomposition approach using the single program, multiple data paradigm. The Message Passing Interface library (MPI) is used for communication of data. An ILU(0) iterative solver is used for smoothing the errors arising within each partition at the different grid levels, and a multi-level synchronization across the computational domain partitions is enforced in order to improve the performance of the parallelized multigrid solver. Two synchronization strategies are evaluated. In the first the synchronization is applied across the multigrid levels during the restriction step in addition to the base level, while in the second the synchronization is enforced during the restriction and prolongation steps. The effect of gathering the coefficients across partitions for the coarsest level is also investigated. Tests on grids up to 800,000 elements are conducted for a number of diffusion and advection problems on up to 20 processors. Results show that synchronization across partitions for multigrid levels plays an essential role in ensuring good scalability. Furthermore, for a large number of partitions, gathering coefficients across partitions is important to ensure a convergence history that is consistent with the sequential solver, thus yielding the same number of iterations for parallel and sequential runs, which is crucial for retaining high scalability. The shadow-to-core elements ratio is also shown to be a good indicator for scalability.

[1]  Chang Sung Kim,et al.  Computational Challenges of Viscous Incompressible Flows , 2005 .

[2]  R. P. Fedorenko.  A relaxation method for solving elliptic difference equations , 1962 .

[3]  Dimitri J. Mavriplis,et al.  A THREE DIMENSIONAL MULTIGRID REYNOLDS-AVERAGED NAVIER-STOKES SOLVER FOR UNSTRUCTURED MESHES , 1994 .

[4]  A. Brandt.  Multi-level adaptive technique (MLAT) for fast numerical solution to boundary value problems , 1973 .

[5]  A. Brandt.  Algebraic multigrid theory: The symmetric case , 1986 .

[6]  Stuart Connell,et al.  A 3D unstructured adaptive multigrid scheme for the Euler equations , 1993 .

[7]  J. W. Ruge,et al.  4. Algebraic Multigrid , 1987 .

[8]  D. Brandt,et al.  Multi-level adaptive solutions to boundary-value problems math comptr , 1977 .

[9]  A. G. Hutton,et al.  THE NUMERICAL TREATMENT OF ADVECTION: A PERFORMANCE COMPARISON OF CURRENT METHODS , 1982 .

[10]  Y. Kallinderis,et al.  New multigrid approach for three-dimensional unstructured, adaptive grids , 1994 .

[11]  Stéphane Lanteri,et al.  Parallel multigrid methods for the calculation of unsteady flows on unstructured grids: algorithmic aspects and parallel performances on clusters of PCs , 2004, Parallel Comput..

[12]  K. Stüben.  Algebraic multigrid (AMG): experiences and comparisons , 1983 .

[13]  Klaus Stüben,et al.  Parallel algebraic multigrid based on subdomain blocking , 2001, Parallel Comput..

[14]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[15]  Manel Soria Guerrero.  Parallel multigrid algorithms for computational fluid dynamics and heat transfer , 2000 .

[16]  R. E. Phillips,et al.  A MULTILEVEL-MULTIGRID TECHNIQUE FOR RECIRCULATING FLOWS , 1985 .

[17]  William P. Timlake,et al.  An Accelerated Relaxation Algorithm for Iterative Solution of Elliptic Equations , 1968 .

[18]  William Gropp,et al.  Parallel computing and domain decomposition , 1992 .

[19]  Y. Hwang.  UNSTRUCTURED ADDITIVE CORRECTION MULTIGRID METHOD FOR THE SOLUTION OF MATRIX EQUATIONS , 1995 .

[20]  S. R. Elias,et al.  AN ADAPTIVE AGGLOMERATION METHOD FOR ADDITIVE CORRECTION MULTIGRID , 1997 .

[21]  S. Acharya,et al.  Comparison of the Piso, Simpler, and Simplec Algorithms for the Treatment of the Pressure-Velocity Coupling in Steady Flow Problems , 1986 .

[22]  F. W. Schmidt,et al.  Multigrid techniques for the solution of the passive scalar advection-diffusion equation , 1985 .

[23]  Performance of algebraic multi‐grid solvers based on unsmoothed and smoothed aggregation schemes , 2001 .

[24]  J. D.,et al.  AGGLOMERATION MULTIGRID FOR VISCOUS TURBULENT FLOWS , 1994 .

[25]  M. Carbonaro,et al.  von Karman Institute for Fluid Dynamics , 2004 .

[26]  R. P. Fedorenko.  The speed of convergence of one iterative process , 1964 .

[27]  Fernando Gustavo Tinetti,et al.  Parallel programming: techniques and applications using networked workstations and parallel computers. Barry Wilkinson, C. Michael Allen , 2000 .

[28]  G. Raithby,et al.  Application of additive correction multigrid to the coupled fluid flow equations , 1988 .

[29]  Y. Joshi,et al.  Algebraic Multigrid Preconditioned Krylov Subspace Methods for Fluid Flow and Heat Transfer on Unstructured Meshes , 2006 .

[30]  Alain Dervieux,et al.  Unstructured multigridding by volume agglomeration: Current status , 1992 .

[31]  R. Białecki.  Applying the boundary element technique to the solution of heat radiation problems in cavities filled by a nongray emitting-absorbing medium , 1991 .

[32]  M. Darwish,et al.  A UNIFIED FORMULATION OF THE SEGREGATED CLASS OF ALGORITHMS FOR FLUID FLOW AT ALL SPEEDS , 2000 .

[33]  Z. Yan.  PARALLEL COMPUTATION OF TURBULENT COMBUSTION AND FLAME SPREAD IN FIRES , 2001 .

[34]  StübenKlaus.  Algebraic multigrid (AMG) , 1983 .

[35]  G. Raithby,et al.  A multigrid method based on the additive correction strategy , 1986 .

[36]  Michel Fortin,et al.  An algebraic multilevel parallelizable preconditioner for large-scale CFD problems , 1997 .

[37]  Yongmo Kim,et al.  PARALLEL UNSTRUCTURED-GRID FINITE-VOLUME METHOD FOR TURBULENT NONPREMIXED FLAMES USING THE FLAMELET MODEL , 2003 .

[38]  PARALLEL COMPUTATION OF TURBULENT COMBUSTION AND FLAME SPREAD IN FIRES , 2001 .

[39]  Christon,et al.  Spatial domain-based parallelism in large-scale, participating-media, radiative transport applications , 1997 .

[40]  S. Patankar.  Numerical Heat Transfer and Fluid Flow , 2018, Lecture Notes in Mechanical Engineering.

[41]  J. Peraire,et al.  A 3D finite element multigrid solver for the Euler equations , 1992 .

[42]  George Karypis,et al.  Multilevel k-way Partitioning Scheme for Irregular Graphs , 1998, J. Parallel Distributed Comput..

[43]  An Enhanced Additive Correction Multigrid Method , 2006 .

[44]  Kumar K. Tamma,et al.  Highly scalable parallel computational models for large-scale RTM process modeling simulations, Part 1: Theoretical formulations and generic design , 1999 .

[45]  Kevin McManus,et al.  A strategy for mapping unstructured mesh computational mechanics programs onto distributed memory parallel architectures , 1996 .

[46]  John G. Georgiadis,et al.  PARALLEL COMPUTATION OF FORCED CONVECTION USING DOMAIN DECOMPOSITION , 1991 .

[47]  Dimitri J. Mavriplis,et al.  Directional Agglomeration Multigrid Techniques for High-Reynolds Number Viscous Flows , 1998 .

[48]  Dimitri J. Mavriplis,et al.  Directional agglomeration multigrid techniques for high Reynolds number viscous flow solvers , 1998 .

[49]  H. M. Bücker.  Iteratively Solving Large Sparse Linear Systems on Parallel Computers , 2002 .

[50]  D. Holmes,et al.  Three-dimensional unstructured adaptive multigrid scheme for the Euler equations , 1994 .

[51]  William Gropp,et al.  High-performance parallel implicit CFD , 2001, Parallel Comput..