Impact of data distribution on the parallel performance of iterative linear solvers with emphasis on CFD of incompressible flows

A parallel data structure that gives optimized memory layout for problems involving iterative solution of sparse linear systems is developed, and its efficient implementation is presented. The proposed method assigns a processor to a problem subdomain, and sorts data based on the shared entries with the adjacent subdomains. Matrix–vector-product communication overhead is reduced and parallel scalability is improved by overlapping inter-processor communications and local computations. The proposed method simplifies the implementation of parallel iterative linear equation solver algorithms and reduces the computational cost of vector inner products and matrix–vector products. Numerical results demonstrate very good performance of the proposed technique.

[1]  T. Tezduyar.  Computation of moving boundaries and interfaces and stabilization parameters , 2003 .

[2]  William Gropp,et al.  PETSc Users Manual Revision 3.4 , 2016 .

[3]  Yuri Bazilevs,et al.  A new preconditioning technique for implicitly coupled multidomain simulations with applications to hemodynamics , 2013 .

[4]  A. Marsden,et al.  A comparison of outlet boundary treatments for prevention of backflow divergence with relevance to blood flow simulations , 2011 .

[5]  Tayfun E. Tezduyar,et al.  Modelling of fluid–structure interactions with the space–time finite elements: Solution techniques , 2007 .

[6]  Matthew G. Knepley,et al.  PETSc Users Manual (Rev. 3.3) , 2013 .

[7]  Howard C. Elman,et al.  Finite Elements and Fast Iterative Solvers: with Applications in Incompressible Fluid Dynamics , 2014 .

[8]  Murat Manguoglu,et al.  Solution of linear systems in arterial fluid mechanics computations with boundary layer mesh refinement , 2010 .

[9]  A. Marsden,et al.  Image-based modeling of hemodynamics in coronary artery aneurysms caused by Kawasaki disease , 2011, Biomechanics and Modeling in Mechanobiology.

[10]  Tayfun E. Tezduyar,et al.  Enhanced-approximation linear solution technique (EALST) , 2004 .

[11]  Tain-Yen Hsia,et al.  A non-discrete method for computation of residence time in fluid mechanics simulations. , 2013, Physics of fluids.

[12]  Tayfun E. Tezduyar,et al.  Enhanced‐discretization successive update method (EDSUM) , 2005 .

[13]  Marek Behr,et al.  Parallel finite-element computation of 3D flows , 1993, Computer.

[14]  A. Sameh,et al.  A nested iterative scheme for computation of incompressible flows in long domains , 2008 .

[15]  Tayfun E. Tezduyar,et al.  Parallel finite element computations in fluid mechanics , 2006 .

[16]  Yuri Bazilevs,et al.  Low entropy data mapping for sparse iterative linear solvers , 2013, XSEDE.

[17]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[18]  David J. Kuck,et al.  On Stable Parallel Linear System Solvers , 1978, JACM.

[19]  Tayfun E. Tezduyar,et al.  Finite elements in fluids: Special methods and enhanced solution techniques , 2007 .

[20]  T. Hughes,et al.  Variational multiscale residual-based turbulence modeling for large eddy simulation of incompressible flows , 2007 .

[21]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[22]  Yuri Bazilevs,et al.  A bi-partitioned iterative algorithm for solving linear systems arising from incompressible flow problems , 2015 .

[23]  Tayfun E. Tezduyar,et al.  Finite element methods for flow problems with moving boundaries and interfaces , 2001 .

[24]  Murat Manguoglu,et al.  A parallel sparse algorithm targeting arterial fluid mechanics computations , 2011 .

[25]  Tayfun E. Tezduyar,et al.  A robust preconditioner for fluid–structure interaction problems , 2005 .

[26]  Alison L. Marsden,et al.  A modular numerical method for implicit 0D/3D coupling in cardiovascular finite element simulations , 2013, J. Comput. Phys..

[27]  Yuri Bazilevs,et al.  Computational Fluid-Structure Interaction: Methods and Applications , 2013 .

[28]  Ahmed H. Sameh,et al.  A parallel hybrid banded system solver: the SPIKE algorithm , 2006, Parallel Comput..

[29]  Murat Manguoglu,et al.  Nested and parallel sparse algorithms for arterial fluid mechanics computations with boundary layer mesh refinement , 2011 .

[30]  A. Sameh,et al.  Preconditioning Techniques for Nonsymmetric Linear Systems in the Computation of Incompressible Flows , 2009 .

[31]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[32]  G. Hulbert,et al.  A generalized-α method for integrating the filtered Navier–Stokes equations with a stabilized finite element method , 2000 .

[33]  Marek Behr,et al.  A new mixed preconditioning method for finite element computations , 1992 .

[34]  Grouped element-by-element iteration schemes for incompressible flow computations , 1989 .

[35]  Mario A. Storti,et al.  Physics based GMRES preconditioner for compressible and incompressible Navier-Stokes equations , 1998 .

[36]  T. Hughes,et al.  Streamline upwind/Petrov-Galerkin formulations for convection dominated flows with particular emphasis on the incompressible Navier-Stokes equations , 1990 .

[37]  S. Mittal,et al.  Incompressible flow computations with stabilized bilinear and linear equal-order-interpolation velocity-pressure elements , 1992 .

[38]  Tayfun E. Tezduyar,et al.  3D Simulation of fluid-particle interactions with the number of particles reaching 100 , 1997 .

[39]  Marek Behr,et al.  Finite element solution strategies for large-scale flow simulations☆ , 1994 .

[40]  Marek Behr,et al.  Implementation of implicit finite element methods for incompressible flows on the CM-5 , 1994 .

[41]  Alison L. Marsden,et al.  Optimization of shunt placement for the Norwood surgery using multi-domain modeling. , 2012, Journal of biomechanical engineering.

[42]  J. G. Kennedy,et al.  Computation of incompressible flows with implicit finite element implementations on the Connection Machine , 1993 .

[43]  D J Kuck,et al.  Parallel Supercomputing Today and the Cedar Approach , 1986, Science.