A Framework for Parallel PDE Solvers on Multiscale Adaptive Cartesian Grids

In many fields of application in science and engineering, the grid-based numerical simulation of partial differential equations leads to new scientific insights due to increasing computing resources, increasing amounts of data, and increasing efficiency of the algorithms used. All three of them facilitate more detailed models and more reliable simulations. Yet, a growing code complexity accompanies this progress. To tackle this complexity, more and more solvers for partial differential equations rely on frameworks. State-of-the-art frameworks have to support multiscale algorithms for arbitrary dimensional problems with dynamic, i.e. changing, discretisations. Despite this flexibility of data and data access, the realisation has to have low memory requirements, as the gap between computing power and memory bandwidth and access speed broadens. It furthermore has to exploit modern computer architectures and has to scale on parallel computers, even if the data structures and the data access pattern change permanently. This thesis presents a framework tackling these challenges with adaptive Cartesian grids resulting from a generalised octree concept. They are traversed with space-filling curves and a small yet fixed number of stacks acting as data containers. Within this context, dynamically changing multiscale grids of arbitrary dimension can be stored with a few bits per datum, whereas classical approaches often require several kilobytes and more flexible data structures entailing a runtime overhead. The thesis formalises the approach and reduces the implementation complexity—the algorithmic principle itself has been well-known for several years—from an exponential to a linear number of containers in the spatial dimension of the problem. A modification of the grid traversal originally following a depth-first order then facilitates a domain decomposition strategy with dynamic load-balancing. The framework is named after the Italian mathematician Giuseppe Peano who discovered the underlying space-filling curve. Its potential is demonstrated via a geometric multigrid solver for the Poisson equation with low memory requirements, good cache hit rates, a posteriori adaptive grids, and a dynamic load balancing— a combination of characteristics rarely found. As the characteristics result from the framework usage, the thesis paves the way to a multiscale computational fluid dynamics application on instationary, hierarchical grids that uses Peano.

[1]  Ulrich Rüde,et al.  Cache Optimization for Structured and Unstructured Grid Multigrid , 2000 .

[2]  F. Harlow,et al.  Numerical Calculation of Time‐Dependent Viscous Incompressible Flow of Fluid with Free Surface , 1965 .

[3]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[4]  William F. Mitchell,et al.  A refinement-tree based partitioning method for dynamic load balancing with adaptively refined grids , 2007, J. Parallel Distributed Comput..

[5]  Georg Stadler,et al.  Scalable adaptive mantle convection simulation on petascale supercomputers , 2008, 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis.

[6]  W. Mitchell The full domain partition approach to distributing adaptive grids , 1998 .

[7]  Graham F. Carey,et al.  Performance analysis of dynamic load balancing algorithms with variable number of processors , 2005, J. Parallel Distributed Comput..

[8]  Donald E. Knuth,et al.  The Genesis of Attribute Grammars , 1990, WAGA.

[9]  F. Frost Report of the president. , 1950, Journal of the National Association of Chiropodists.

[10]  Hanan Samet,et al.  The Quadtree and Related Hierarchical Data Structures , 1984, CSUR.

[11]  Hans-Joachim Bungartz,et al.  A precompiler to reduce the memory footprint of multiscale PDE solvers in C++ , 2010, Future Gener. Comput. Syst..

[12]  Michael Griebel,et al.  Hash-Storage Techniques for Adaptive Multilevel Solvers and Their Domain Decomposition Parallelizati , 1998 .

[13]  Martin Fowler,et al.  Patterns of Enterprise Application Architecture , 2002 .

[14]  Michael Lindenbaum,et al.  On the metric properties of discrete space-filling curves , 1996, IEEE Trans. Image Process..

[15]  C. Weiss,et al.  Memory Characteristics of Iterative Methods , 1999, ACM/IEEE SC 1999 Conference (SC'99).

[16]  Donald E. Knuth,et al.  The Art of Computer Programming, Volumes 1-3 Boxed Set , 1998 .

[17]  Michael Griebel,et al.  Parallel multigrid in an adaptive PDE solver based on hashing and space-filling curves , 1999, Parallel Comput..

[18]  Andreas Krahnke,et al.  Adaptive Verfahren höherer Ordnung auf cache-optimalen Datenstrukturen für dreidimensionale Probleme , 2005 .

[19]  Miriam Mehl,et al.  A Cache-Aware Algorithm for PDEs on Hierarchical Data Structures Based on Space-Filling Curves , 2006, SIAM J. Sci. Comput..

[20]  Bjarne Stroustrup Die C++-Programmiersprache , 2011 .

[21]  Wolfgang E. Nagel,et al.  Optimizing OpenMP Parallelized DGEMM Calls on SGI Altix 3700 , 2006, Euro-Par.

[22]  Hans-Joachim Bungartz,et al.  Numerical Simulation of Particle Transport in a Drift Ratchet , 2008, SIAM J. Sci. Comput..

[23]  Ralph Johnson,et al.  design patterns elements of reusable object oriented software , 2019 .

[24]  M. Berger,et al.  Adaptive mesh refinement for hyperbolic partial differential equations , 1982 .

[25]  Bharat K. Soni,et al.  Handbook of Grid Generation , 1998 .

[26]  Frank Günther,et al.  Eine cache-optimale Implementierung der Finite-Elemente-Methode , 2004 .

[27]  John von Neumann,et al.  First draft of a report on the EDVAC , 1993, IEEE Annals of the History of Computing.

[28]  V. Gregory Weirs,et al.  Adaptive Mesh Refinement - Theory and Applications , 2008 .

[29]  Irad Yavneh,et al.  Why Multigrid Methods Are So Efficient , 2006, Computing in Science & Engineering.

[30]  Benjamin Karl Bergen,et al.  Hierarchical hybrid grids: data structures and core algorithms for efficient finite element simulations on supercomputers = Hierarchische hybride Gitter , 2005 .

[31]  James C. Browne,et al.  A common data manage-ment infrastructure for parallel adaptive algorithms for PDE solutions , 1997 .

[32]  William F. Mitchell,et al.  A PARALLEL MULTIGRID METHOD USING THE FULL DOMAIN PARTITION , 1997 .

[33]  Michael Bader,et al.  Memory efficient adaptive mesh generation and implementation of multigrid algorithms using Sierpinski curves , 2008, Int. J. Comput. Sci. Eng..

[34]  Thomas Huckle,et al.  Compact Fourier Analysis for Designing Multigrid Methods , 2008, SIAM J. Sci. Comput..

[35]  William Gropp,et al.  High-performance parallel implicit CFD , 2001, Parallel Comput..

[36]  Anton Christian Frank Organisationsprinzipien zur Integration von geometrischer Modellierung, numerischer Simulation und Visualisierung , 2000 .

[37]  David E. Keyes,et al.  Four Horizons for Enhancing the Performance of Parallel Simulations Based on Partial Differential Equations , 2000, Euro-Par.

[38]  Tobias Neckel The PDE Framework Peano: An Environment for Efficient Flow Simulations , 2009 .

[39]  Hans-Joachim Bungartz,et al.  DaStGen-A Data Structure Generator for Parallel C++ HPC Software , 2008, ICCS.

[40]  Mark de Berg,et al.  Computational geometry: algorithms and applications , 1997 .

[41]  Hanan Samet,et al.  The Design and Analysis of Spatial Data Structures , 1989 .

[42]  Hans-Joachim Bungartz,et al.  A Parallel Adaptive Cartesian PDE Solver Using Space-Filling Curves , 2006, Euro-Par.

[43]  Andreas Dedner,et al.  A generic grid interface for parallel and adaptive scientific computing. Part II: implementation and tests in DUNE , 2008, Computing.

[44]  Michael Griebel,et al.  Multilevelmethoden als Iterationsverfahren über Erzeugendensystemen , 1994 .

[45]  Ullrich Rüde Mathematical and Computational Techniques for Multilevel Adaptive Methods , 1987 .

[46]  Ralf-Peter Mundani Hierarchische Geometriemodelle zur Einbettung verteilter Simulationsaufgaben , 2006 .

[47]  Georg Stadler,et al.  Towards adaptive mesh PDE simulations on petascale computers , 2008 .

[48]  Martin C. Rinard,et al.  Recursion Unrolling for Divide and Conquer Programs , 2000, LCPC.

[49]  M. Broy Informatik Eine grundlegende Einführung , 1998 .

[50]  Markus Kowarschik,et al.  An Overview of Cache Optimization Techniques and Cache-Aware Numerical Algorithms , 2002, Algorithms for Memory Hierarchies.

[51]  Vladimir D. Liseikin,et al.  Grid Generation Methods , 1999 .

[52]  Andreas Dedner,et al.  A generic grid interface for parallel and adaptive scientific computing. Part I: abstract framework , 2008, Computing.

[53]  Jens-Michael Wierum,et al.  On the Quality of Partitions Based on Space-Filling Curves , 2002, International Conference on Computational Science.

[54]  Ulrich Rüde,et al.  High Performance Multigrid on Current Large Scale Parallel Computers , 2008, PASA.