The impact of enabling multiple subdomains per MPI process in the TFETI domain decomposition method

Abstract The paper deals with handling multiple subdomains per computational core in the PERMON toolbox, namely in the PermonFLLOP module, to fully exploit the potential of the Total Finite Element Tearing and Interconnecting (TFETI) domain decomposition method (DDM). Most authors researching FETI methods present weak parallel scalability with one subdomain assigned to each computational core, and call it just parallel scalability. Here we present an extension showing the data of more than one subdomain being held by each MPI process. Numerical experiments demonstrate the theoretically supported fact that for the given problem size and number of processors, the increased number of subdomains leads to better conditioning of the system operator, and hence faster convergence. Moreover, numerical, memory, strong parallel, and weak parallel scalability is reported, and optimal numbers of subdomains per core are examined. Finally, new PETSc matrix types dealing with the aforementioned extension are introduced.

[1]  Charbel Farhat,et al.  An Unconventional Domain Decomposition Method for an Efficient Parallel Solution of Large-Scale Finite Element Systems , 1992, SIAM J. Sci. Comput..

[2]  Martin Cermák,et al.  Total FETI domain decomposition method and its massively parallel implementation , 2013, Adv. Eng. Softw..

[3]  Vaclav Hapla,et al.  Implementation of the efficient communication layer for the highly parallel total FETI and hybrid total FETI solvers , 2016, Parallel Comput..

[4]  C. Farhat,et al.  Optimal convergence properties of the FETI domain decomposition method , 1994 .

[5]  Alena Vasatová,et al.  Solving Contact Mechanics Problems with PERMON , 2015, HPCSE.

[6]  Vaclav Hapla,et al.  Use of Direct Solvers in TFETI Massively Parallel Implementation , 2012, PARA.

[7]  Zdenek Dostál,et al.  Cholesky decomposition of a positive semidefinite matrix with known kernel , 2011, Appl. Math. Comput..

[8]  Jaroslav Kruis.  Domain Decomposition Methods for Distributed Computing , 2007 .

[9]  Daniel Rixen,et al.  Application of the FETI method to ASCI problems—scalability results on 1000 processors and discussion of highly heterogeneous problems , 2000 .

[10]  C. Farhat,et al.  The second generation FETI methods and their application to the parallel solution of large-scale linear and geometrically non-linear structural analysis problems , 2000 .

[11]  Daniel Rixen,et al.  THEORETICAL COMPARISON OF THE FETI AND ALGEBRAICALLY PARTITIONED FETI METHODS, AND PERFORMANCE COMPARISONS WITH A DIRECT SPARSE SOLVER , 1999 .

[12]  O. Widlund,et al.  FETI and Neumann--Neumann Iterative Substructuring Methods: Connections and New Results , 1999 .

[13]  D. Rixen,et al.  FETI‐DP: a dual–primal unified FETI method—part I: A faster alternative to the two‐level FETI method , 2001 .

[14]  CLARK R. DOHRMANN,et al.  A Preconditioner for Substructuring Based on Constrained Energy Minimization , 2003, SIAM J. Sci. Comput..

[15]  Alena Vasatová,et al.  Parallel Implementation of the FETI DDM Constraint Matrix on Top of PETSc for the PermonFLLOP Package , 2015, PPAM.

[16]  Alena Vasatová,et al.  Comparison of different FETI preconditioners for elastoplasticity , 2017, Comput. Math. Appl..

[17]  Vaclav Hapla,et al.  TFETI Coarse Space Projectors Parallelization Strategies , 2011, PPAM.

[18]  C. Farhat,et al.  A method of finite element tearing and interconnecting and its parallel solution algorithm , 1991 .

[19]  Z. Dostál,et al.  Total FETI—an easier implementable variant of the FETI method for numerical solution of elliptic PDE , 2006 .

[20]  A. Klawonn,et al.  Highly scalable parallel domain decomposition methods with an application to biomechanics , 2010 .