Flexible and Scalable Particle‐in‐Cell Methods With Adaptive Mesh Refinement for Geodynamic Computations

Particle-in-cell methods couple mesh-based methods for the solution of continuum mechanics problems, with the ability to advect and evolve particles. They have a long history and many applications in scientific computing. However, they have most often only been implemented for either sequential codes, or parallel codes with static meshes that are statically partitioned. In contrast, many mesh-based codes today use adaptively changing, dynamically partitioned meshes, and can scale to thousands or tens of thousands of processors. Consequently, there is a need to revisit the data structures and algorithms necessary to use particle methods with modern, mesh-based methods. Here we review commonly encountered requirements of particle-in-cell methods, and describe efficient ways to implement them in the context of large-scale parallel finite-element codes that use dynamically changing meshes. We also provide practical experience for how to address bottlenecks that impede the efficient implementation of these algorithms and demonstrate with numerical tests both that our algorithms can be implemented with optimal complexity and that they are suitable for very large-scale, practical applications. We provide a reference implementation in ASPECT, an open source code for geodynamic mantle-convection simulations built on the deal.II library.

[1]  Carsten Burstedde,et al.  Recursive Algorithms for Distributed Forests of Octrees , 2014, SIAM J. Sci. Comput..

[2]  E. Hairer,et al.  Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems , 2010 .

[3]  Mohammad Mirzadeh,et al.  Parallel level-set methods on adaptive tree-based grids , 2016, J. Comput. Phys..

[4]  James E. Guilkey,et al.  A weighted least squares particle‐in‐cell method for solid mechanics , 2011 .

[5]  James Jackson,et al.  The relationship between strain rates, crustal thickening, palaeomagnetism, finite strain and fault movements within a deforming zone , 1983 .

[6]  Carsten Burstedde Parallel Tree Algorithms for AMR and Non-Standard Data Access , 2020, ACM Trans. Math. Softw..

[7]  Wei Leng,et al.  Implementation and application of adaptive mesh refinement for thermochemical mantle convection studies , 2011 .

[8]  J. Oden,et al.  A Posteriori Error Estimation in Finite Element Analysis , 2000 .

[9]  Martin Kronbichler,et al.  Algorithms and data structures for massively parallel generic adaptive finite element codes , 2011, ACM Trans. Math. Softw..

[10]  Graham F. Carey,et al.  Computational grids : generation, adaptation, and solution strategies , 1997 .

[11]  Maria Seton,et al.  Global continental and ocean basin reconstructions since 200 Ma , 2012 .

[12]  D. May,et al.  Discretization Errors in the Hybrid Finite Element Particle-in-cell Method , 2014, Pure and Applied Geophysics.

[13]  Ken Kennedy,et al.  Improving Memory Hierarchy Performance for Irregular Applications Using Data and Computation Reorderings , 2001, International Journal of Parallel Programming.

[14]  William Gropp,et al.  PETSc Users Manual Revision 3.4 , 2016 .

[15]  Eugenio Aulisa,et al.  A particle tracking algorithm for parallel finite element applications , 2017 .

[16]  B. Steinberger,et al.  Models of large‐scale viscous flow in the Earth's mantle with constraints from mineral physics and surface observations , 2006 .

[17]  Boris J. P. Kaus,et al.  Comparison of Eulerian and Lagrangian numerical techniques for the Stokes equations in the presence of strongly varying viscosity , 2008 .

[18]  Jonathan M. Robey,et al.  New numerical approaches for modeling thermochemical convection in a compositionally stratified fluid , 2017, 1703.02202.

[19]  David A. Yuen,et al.  Characteristics-based marker-in-cell method with conservative finite-differences schemes for modeling geological flows with strongly variable transport properties , 2003 .

[20]  David Wells,et al.  The deal.II library, version 8.5 , 2013, J. Num. Math..

[21]  David Wells,et al.  The deal.II Library, Version 8.4 , 2016, J. Num. Math..

[22]  Marie-Pierre Doin,et al.  A comparison of methods for the modeling of thermochemical convection , 1997 .

[23]  Hans Muhlhaus,et al.  A Lagrangian integration point finite element method for large deformation modeling of viscoelastic geomaterials , 2003 .

[24]  Harold A. Buetow,et al.  Adaptive finite element methods for differential equations , 2003, Lectures in mathematics.

[25]  Carsten Burstedde,et al.  p4est: Scalable Algorithms for Parallel Adaptive Mesh Refinement on Forests of Octrees , 2011, SIAM J. Sci. Comput..

[26]  Thorsten W. Becker,et al.  Comparison of azimuthal seismic anisotropy from surface waves and finite strain from global mantle-circulation models , 2003 .

[27]  S. V. Sobolev,et al.  SLIM3D: A tool for three-dimensional thermomechanical modeling of lithospheric deformation with elasto-visco-plastic rheology , 2008 .

[28]  Wolfgang Bangerth,et al.  High accuracy mantle convection simulation through modern numerical methods , 2012 .

[29]  David A. Yuen,et al.  Extended-Boussinesq thermal–chemical convection with moving heat sources and variable viscosity , 2000 .

[30]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[31]  J. Oden,et al.  A Posteriori Error Estimation in Finite Element Analysis: Oden/A Posteriori , 2000 .

[32]  W. Bangerth,et al.  deal.II—A general-purpose object-oriented finite element library , 2007, TOMS.

[33]  Martin Kronbichler,et al.  High accuracy mantle convection simulation through modern numerical methods , 2012 .

[34]  Francis H Harlow,et al.  The particle-in-cell method for numerical solution of problems in fluid dynamics , 1962 .

[35]  Takashi Nakagawa,et al.  Incorporating self‐consistently calculated mineral physics into thermochemical mantle convection simulations in a 3‐D spherical shell and its influence on seismic anomalies in Earth's mantle , 2009 .

[36]  A. Poliakov,et al.  Diapirism and topography , 1992 .

[37]  Shijie Zhong,et al.  Thermochemical structures within a spherical mantle: Superplumes or piles? , 2004 .

[38]  Jean M. Sexton,et al.  Nyx: A MASSIVELY PARALLEL AMR CODE FOR COMPUTATIONAL COSMOLOGY , 2013, J. Open Source Softw..

[39]  J. van Hunen,et al.  Advantages of a conservative velocity interpolation (CVI) scheme for particle‐in‐cell methods with application in geodynamic modeling , 2015, Geochemistry, geophysics, geosystems : G(3).