Pragmatic solvers for 3D Stokes and elasticity problems with heterogeneous coefficients: evaluating modern incomplete LDL T preconditioners

Abstract. The need to solve large saddle point systems within computational Earth sciences is ubiquitous. Physical processes giving rise to these systems include porous flow (the Darcy equations), poroelasticity, elastostatics, and highly viscous flows (the Stokes equations). The numerical solution of saddle point systems is non-trivial since the operators are indefinite. Primary tools to solve such systems are direct solution methods (exact triangular factorization) or Approximate Block Factorization (ABF) preconditioners. While ABF solvers have emerged as the state-of-the-art scalable option, they are invasive solvers requiring splitting of pressure and velocity degrees of freedom, a multigrid hierarchy with tuned transfer operators and smoothers, machinery to construct complex Schur complement preconditioners, and the expertise to select appropriate parameters for a given coefficient regime – they are far from being "black box" solvers. Modern direct solvers, which robustly produce solutions to almost any system, do so at the cost of rapidly growing time and memory requirements for large problems, especially in 3D. Incomplete LDL (ILDL) factorizations, with symmetric maximum weighted matching preprocessing, used as preconditioners for Krylov (iterative) methods, have emerged as an efficient means to solve indefinite systems. These methods have been developed within the numerical linear algebra community but have yet to become widely used in non-trivial applications, despite their practical potential; they can be used whenever a direct solver can, only requiring an assembled operator, yet can offer comparable or superior to performance, with the added benefit of having a much lower memory footprint. In comparison to ABF solvers, they only require the specification of a drop tolerance and thus provide an easy-to-use addition to the solver toolkit for practitioners. Here, we present solver experiments employing incomplete LDL factorization with symmetric maximum weighted matching preprocessing to precondition operators, and compare these to direct solvers and ABF-preconditioned iterative solves. To ensure the comparison study is meaningful for Earth scientists, we utilize matrices arising from two prototypical problems, namely Stokes flow and quasi-static (linear) elasticity, discretized using standard mixed finite element spaces. Our test suite targets problems with large coefficient discontinuities across non-grid-aligned interfaces, which represent a common, challenging-for-solvers, scenario in Earth science applications. Our results show: (i) as the coefficient structure is made increasingly challenging (high contrast, complex topology), the ABF solver can break down, becoming less efficient than the ILDL solver before breaking down entirely; (ii) ILDL is robust, with a time-to-solution that is largely independent of the coefficient topology and mildly dependent on the coefficient contrast; (iii) the time-to-solution obtained using ILDL is typically faster than that obtained from a direct solve, beyond 10^5 unknowns; (iv) ILDL always uses less memory than a direct solve.

[1]  M. Benzi,et al.  Some Preconditioning Techniques for Saddle Point Problems , 2008 .

[2]  Georg Stadler,et al.  Weighted BFBT Preconditioner for Stokes Flow Problems with Highly Heterogeneous Viscosity , 2016, SIAM J. Sci. Comput..

[3]  Marcus J. Grote,et al.  Algebraic Multilevel Preconditioner for the Helmholtz Equation in Heterogeneous Media , 2009, SIAM J. Sci. Comput..

[4]  Michael B. Giles,et al.  Preconditioned iterative solution of the 2D Helmholtz equation , 2002 .

[5]  Iain S. Duff,et al.  Strategies for Scaling and Pivoting for Sparse Symmetric Indefinite Problems , 2005, SIAM J. Matrix Anal. Appl..

[6]  Rudolf A. Römer,et al.  On large‐scale diagonalization techniques for the Anderson model of localization , 2005, SIAM J. Sci. Comput..

[7]  A. George Nested Dissection of a Regular Finite Element Mesh , 1973 .

[8]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[9]  Tamara G. Kolda,et al.  An overview of the Trilinos project , 2005, TOMS.

[10]  Maxim A. Olshanskii,et al.  An Iterative Method for the Stokes-Type Problem with Variable Viscosity , 2009, SIAM J. Sci. Comput..

[11]  Yousef Saad,et al.  Parallel Iterative Methods for Sparse Linear Systems , 2001 .

[12]  Jennifer A. Scott,et al.  On Signed Incomplete Cholesky Factorization Preconditioners for Saddle-Point Systems , 2014, SIAM J. Sci. Comput..

[13]  M. Benzi Preconditioning techniques for large linear systems: a survey , 2002 .

[14]  David A. Yuen,et al.  Characteristics-based marker-in-cell method with conservative finite-differences schemes for modeling geological flows with strongly variable transport properties , 2003 .

[15]  Jonathan J. Hu,et al.  Parallel multigrid smoothing: polynomial versus Gauss--Seidel , 2003 .

[16]  Jonas Thies,et al.  A Robust Two-Level Incomplete Factorization for (Navier-)Stokes Saddle Point Matrices , 2010, SIAM J. Matrix Anal. Appl..

[17]  Andrew J. Wathen,et al.  Analysis of Preconditioners for Saddle-Point Problems , 2004, SIAM J. Sci. Comput..

[18]  Enrique S. Quintana-Ortí,et al.  Leveraging Data-Parallelism in ILUPACK using Graphics Processors , 2014, 2014 IEEE 13th International Symposium on Parallel and Distributed Computing.

[19]  J. Munkres ALGORITHMS FOR THE ASSIGNMENT AND TRANSIORTATION tROBLEMS* , 1957 .

[20]  Shiwen He,et al.  SYM-ILDL , 2015, ArXiv.

[21]  D. Rixen,et al.  FETI‐DP: a dual–primal unified FETI method—part I: A faster alternative to the two‐level FETI method , 2001 .

[22]  J. Bunch,et al.  Some stable methods for calculating inertia and solving symmetric linear systems , 1977 .

[23]  Howard C. Elman,et al.  Preconditioning for the Steady-State Navier-Stokes Equations with Low Viscosity , 1999, SIAM J. Sci. Comput..

[24]  Olaf Schenk,et al.  Fast Methods for Computing Selected Elements of the Green's Function in Massively Parallel Nanoelectronic Device Simulations , 2013, Euro-Par.

[25]  Constantine Bekas,et al.  An extreme-scale implicit solver for complex PDEs: highly heterogeneous flow in earth's mantle , 2015, SC15: International Conference for High Performance Computing, Networking, Storage and Analysis.

[26]  Matthew G. Knepley,et al.  Extreme-Scale Multigrid Components within PETSc , 2016, PASC.

[27]  Franco Brezzi Michel Fortin,et al.  Mixed and Hybrid Finite Element Methods (Springer Series in Computational Mathematics) , 1991 .

[28]  Gene H. Golub,et al.  Numerical solution of saddle point problems , 2005, Acta Numerica.

[29]  Y. Saad,et al.  Experimental study of ILU preconditioners for indefinite matrices , 1997 .

[30]  I. Duff,et al.  The factorization of sparse symmetric indefinite matrices , 1991 .

[31]  Yousef Saad,et al.  ILUT: A dual threshold incomplete LU factorization , 1994, Numer. Linear Algebra Appl..

[32]  Jed Brown,et al.  pTatin3D: High-Performance Methods for Long-Term Lithospheric Dynamics , 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

[33]  Olaf Schenk,et al.  Weighted Matchings for Preconditioning Symmetric Indefinite Linear Systems , 2006, SIAM J. Sci. Comput..

[34]  D. May,et al.  Preconditioned iterative methods for Stokes flow problems arising in computational geodynamics , 2008 .

[35]  Enrique S. Quintana-Ortí,et al.  Communication in task‐parallel ILU‐preconditioned CG solvers using MPI + OmpSs , 2017, Concurr. Comput. Pract. Exp..

[36]  Olaf Schenk,et al.  Solving unsymmetric sparse systems of linear equations with PARDISO , 2004, Future Gener. Comput. Syst..

[37]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[38]  Dave A. May,et al.  A scalable, matrix-free multigrid preconditioner for finite element discretizations of heterogeneous Stokes flow , 2015 .

[39]  R. Freund,et al.  QMR: a quasi-minimal residual method for non-Hermitian linear systems , 1991 .

[40]  Chuan Lu,et al.  Simulating subsurface flow and transport on ultrascale computers using PFLOTRAN , 2007 .

[41]  Enrique S. Quintana-Ortí,et al.  A Data-Parallel ILUPACK for Sparse General and Symmetric Indefinite Linear Systems , 2016, Euro-Par Workshops.

[42]  Marcin Dabrowski,et al.  MILAMIN: MATLAB‐based finite element method solver for large problems , 2008 .

[43]  Simon Scheidegger,et al.  Large-scale Sparse Inverse Covariance Matrix Estimation , 2019, SIAM J. Sci. Comput..

[44]  Nicholas I. M. Gould,et al.  A numerical evaluation of sparse direct solvers for the solution of large sparse symmetric linear systems of equations , 2007, TOMS.

[45]  Enrique S. Quintana-Ortí,et al.  Exploiting task and data parallelism in ILUPACK's preconditioned CG solver on NUMA architectures and many-core accelerators , 2016, Parallel Comput..

[46]  Timothy A. Davis,et al.  Algorithm 832: UMFPACK V4.3---an unsymmetric-pattern multifrontal method , 2004, TOMS.

[47]  Timothy A. Davis,et al.  A survey of direct methods for sparse linear systems , 2016, Acta Numerica.

[48]  A. Neumaier,et al.  A NEW PIVOTING STRATEGY FOR GAUSSIAN ELIMINATION , 1996 .

[49]  Yousef Saad,et al.  Multilevel Preconditioners Constructed From Inverse-Based ILUs , 2005, SIAM J. Sci. Comput..

[50]  James Demmel,et al.  SuperLU_DIST: A scalable distributed-memory sparse direct solver for unsymmetric linear systems , 2003, TOMS.

[51]  Georg Stadler,et al.  Solution of Nonlinear Stokes Equations Discretized By High-Order Finite Elements on Nonconforming and Anisotropic Meshes, with Application to Ice Sheet Dynamics , 2014, SIAM J. Sci. Comput..

[52]  Iain S. Duff,et al.  The Design and Use of Algorithms for Permuting Large Entries to the Diagonal of Sparse Matrices , 1999, SIAM J. Matrix Anal. Appl..