Parallel iterative solvers for finite-element methods using an OpenMP/MPI hybrid programming model on the Earth Simulator

An efficient parallel iterative method for finite-element method has been developed for symmetric multiprocessor (SMP) cluster architectures with vector processors such as the Earth Simulator. The method is based on a three-level hybrid parallel programming model, including message passing for inter-SMP node communication, loop directives by OpenMP for intra-SMP node parallelization and vectorization for each processing element (PE). Simple 3D linear elastic problems with more than 2.2x10^9 DOF have been solved using 3x3 block ICCG(0) method with additive Schwarz domain decomposition and PDJDS/CM-RCM reordering on 176 nodes of the Earth Simulator, achieving performance of 3.80 TFLOPS. Furthermore, effect of color number in reordering has been evaluated on various types of computers.

[1]  Hiroshi Okuda,et al.  Parallel Iterative Solvers for Unstructured Grids Using an OpenMP/MPI Hybrid Programming Model for the GeoFEM Platform on SMP Cluster Architectures , 2002, ISHPC.

[2]  Rolf Rabenseifner,et al.  Communication Bandwidth of Parallel Programming Models on Hybrid Architectures , 2009, ISHPC.

[3]  M. Jahed Djomehri,et al.  Hybrid MPI+OpenMP Programming of an Overset CFD Solver and Performance Investigations , 2002 .

[4]  Franck Cappello,et al.  MPI versus MPI+OpenMP on the IBM SP for the NAS Benchmarks , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[5]  Robert D. Falgout,et al.  Multigrid on massively parallel architectures , 2000 .

[6]  Jack J. Dongarra,et al.  Solving linear systems on vector and shared memory computers , 1990 .

[7]  Leonid Oliker,et al.  Effects of Ordering Strategies and Programming Paradigms on Sparse Matrix Computations , 2013, SIAM Rev..

[8]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[9]  Barry F. Smith,et al.  Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations , 1996 .

[10]  K. Nakajima.  Parallel Iterative Solvers of GeoFEM with Selective Blocking Preconditioning for Nonlinear Contact Problems on the Earth Simulator , 2003, ACM/IEEE SC 2003 Conference (SC'03).

[11]  Richard Barrett,et al.  Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods , 1994, Other Titles in Applied Mathematics.

[12]  Kengo Nakajima,et al.  Parallel Iterative Solvers with Localized ILU Preconditioning for Unstructured Grids on Workstation Clusters , 1999 .

[13]  Genki Yagawa,et al.  GeoFEM: High Performance Parallel FEM for Solid Earth , 1999, HPCN Europe.