An extreme-scale implicit solver for complex PDEs: highly heterogeneous flow in earth's mantle

Mantle convection is the fundamental physical process within earth's interior responsible for the thermal and geological evolution of the planet, including plate tectonics. The mantle is modeled as a viscous, incompressible, non-Newtonian fluid. The wide range of spatial scales, extreme variability and anisotropy in material properties, and severely nonlinear rheology have made global mantle convection modeling with realistic parameters prohibitive. Here we present a new implicit solver that exhibits optimal algorithmic performance and is capable of extreme scaling for hard PDE problems, such as mantle convection. To maximize accuracy and minimize runtime, the solver incorporates a number of advances, including aggressive multi-octree adaptivity, mixed continuous-discontinuous discretization, arbitrarily-high-order accuracy, hybrid spectral/geometric/algebraic multigrid, and novel Schur-complement preconditioning. These features present enormous challenges for extreme scalability. We demonstrate that---contrary to conventional wisdom---algorithmically optimal implicit solvers can be designed that scale out to 1.5 million cores for severely nonlinear, ill-conditioned, heterogeneous, and anisotropic PDEs.

[1]  Georg Stadler,et al.  The Dynamics of Plate Tectonics and Mantle Flow: From Local to Global Scales , 2010, Science.

[2]  Georg Stadler,et al.  Slab stress and strain rate as constraints on global mantle flow , 2010 .

[3]  D. May,et al.  Preconditioned iterative methods for Stokes flow problems arising in computational geodynamics , 2008 .

[4]  Wolfgang Bangerth,et al.  High accuracy mantle convection simulation through modern numerical methods , 2012 .

[5]  Georg Stadler,et al.  Multi‐scale dynamics and rheology of mantle flow with plates , 2011 .

[6]  Santiago Badia,et al.  A Highly Scalable Parallel Implementation of Balancing Domain Decomposition by Constraints , 2014, SIAM J. Sci. Comput..

[7]  Shijie Zhong,et al.  Controls on trench topography from dynamic models of subducted slabs , 1994 .

[8]  Hari Sundar,et al.  Parallel geometric-algebraic multigrid on unstructured forests of octrees , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[9]  Carsten Burstedde,et al.  Recursive Algorithms for Distributed Forests of Octrees , 2014, SIAM J. Sci. Comput..

[10]  Teh-Ru Alex Song,et al.  Large Trench-Parallel Gravity Variations Predict Seismogenic Behavior in Subduction Zones , 2003, Science.

[11]  P. Fischer,et al.  High-Order Methods for Incompressible Fluid Flow , 2002 .

[12]  Nectarios Koziris,et al.  An Extended Compression Format for the Optimization of Sparse Matrix-Vector Multiplication , 2013, IEEE Transactions on Parallel and Distributed Systems.

[13]  Carsten Burstedde,et al.  p4est: Scalable Algorithms for Parallel Adaptive Mesh Refinement on Forests of Octrees , 2011, SIAM J. Sci. Comput..

[14]  Chenjie Yu,et al.  Evaluating Power-Monitoring Capabilities on IBM Blue Gene/P and Blue Gene/Q , 2012, 2012 IEEE International Conference on Cluster Computing.

[15]  Timothy C. Warburton,et al.  Extreme-Scale AMR , 2010, 2010 ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis.

[16]  Donald W. Forsyth,et al.  On the Relative Importance of the Driving Forces of Plate Motion , 1975 .

[17]  Bradford H. Hager,et al.  A simple global model of plate dynamics and mantle convection , 1981 .

[18]  Barbara I. Wohlmuth,et al.  Performance and Scalability of Hierarchical Hybrid Multigrid Solvers for Stokes Systems , 2015, SIAM J. Sci. Comput..

[19]  Gabriel Wittum,et al.  A massively parallel geometric multigrid solver on hierarchically distributed grids , 2013, Comput. Vis. Sci..

[20]  Martin Kronbichler,et al.  High accuracy mantle convection simulation through modern numerical methods , 2012 .

[21]  Nikolaus A. Adams,et al.  11 PFLOP/s simulations of cloud cavitation collapse , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[22]  Homer F. Walker,et al.  Choosing the Forcing Terms in an Inexact Newton Method , 1996, SIAM J. Sci. Comput..

[23]  Georg Stadler,et al.  Adjoint-based estimation of plate coupling in a non-linear mantle flow model: Theory and examples , 2015 .

[24]  Hiroo Kanamori,et al.  Seismic coupling and uncoupling at subduction zones , 1983 .

[25]  Robert D. Falgout,et al.  Scaling Hypre's Multigrid Solvers to 100, 000 Cores , 2011, High-Performance Scientific Computing.

[26]  John N. Shadid,et al.  Block Preconditioners Based on Approximate Commutators , 2005, SIAM J. Sci. Comput..

[27]  Georg Stadler,et al.  Scalable adaptive mantle convection simulation on petascale supercomputers , 2008, 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis.

[28]  Jack Dongarra,et al.  Applied Mathematics Research for Exascale Computing , 2014 .

[29]  Jaime Campos,et al.  The seismic coupling of subduction zones revisited , 2012 .

[30]  Giorgio Ranalli,et al.  Rheology of the earth , 1987 .

[31]  Gene H. Golub,et al.  Numerical solution of saddle point problems , 2005, Acta Numerica.