Constructing a Supercomputing Framework by Using Python for Hybrid Parallelism and GPU Cluster

This paper reports the construction of a new software framework, SOLVCON, for high-fidelity solutions of linear and nonlinear hyperbolic partial differential equations (PDEs). Applications of SOLVCON include gas dynamics, aero-acoustics, stress waves in solids, electromagnetic waves, etc. The default numerical scheme employed in SOLVCON is the space-time Conservation Element and Solution Element (CESE) method. SOLVCON is organized based on the two-loop structure found in all finite-volume methods for time-marching solutions: (i) the outer temporal loop for time-marching, and (ii) the inner spatial loops for calculating flux conservation. Built as a software framework that enforces inversion of control, solver kernels in SOLVCON are completely segregated from supportive functionalities and are made pluggable to the framework. As such, SOLVCON is inherently a multi-physics software package. SOLVCON also aims at reducing turnaround time for high-resolution calculations by incorporating hybrid parallelism, i.e., simultaneously using shared- and distributed-memory parallel computing. The code for General-Purpose Graphic Processing Unit (GPGPU) and multi-threaded computing are modularized as optional parts of the solver kernels. Based on the two-loop algorithm, the software structure of SOLVCON has been organized into 5 layers of Python modules. To handle large data sets associated with high-fidelity solutions of PDEs, special modules were developed for parallel input and output (I/O) and in situ visualization. SOLVCON is developed by using Python programming language because it is efficient to be coded and maintained and its unique capability of interfacing with other programming languages for high-performance computing (HPC). By using SOLVCON for typical computational tasks, 99% of the execution time is used in spatial loops, which are implemented by using C with multi-threading computing, and/or CUDA for GPGPU computing. The use of Python does not impair the performance. To model complex geometry, SOLVCON uses unstructured mesh consisting of mixed elements, i.e., triangles and quadrilaterals for two-dimensional problems, and tetrahedra, prisms, and pyramids for three-dimensional problems. SOLVCON has been routinely used for computational tasks running on clusters composed of more than a thousand CPUs. For a supersonic cross flows passing a jet, a benchmarking CFD problem, with 1.3 billion degrees of freedom, SOLVCON delivers the full transient results, including visualized results, in 3 days. SOLVCON has been open-sourced and released (http://solvcon.net) under the term of GNU General Public License.

[1]  Sin-Chung Chang,et al.  The method of space-time conservation element and solution element-applications to one-dimensional and two-dimensional time-marching flow problems , 1995 .

[2]  R. LeVeque.  Finite Volume Methods for Hyperbolic Problems: Characteristics and Riemann Problems for Linear Hyperbolic Equations , 2002 .

[3]  Sin-Chung Chang,et al.  On an origin of numerical diffusion: Violation of invariance under space-time inversion , 1992 .

[4]  B. Auld.  Acoustic fields and waves in solids. Vol. 1 , 1990 .

[5]  Xing Cai,et al.  On the performance of the Python programming language for serial and parallel scientific computations , 2005, Sci. Program..

[6]  A. Kulikovskii,et al.  Mathematical Aspects of Numerical Solution of Hyperbolic Systems , 1998, physics/9807053.

[7]  Yan Shi,et al.  The finite-volume time-domain algorithm using least square method in solving Maxwell's equations , 2007, J. Comput. Phys..

[8]  Wai Ming To,et al.  A brief description of a new numerical framework for solving conservation laws — The method of space-time conservation element and solution element , 1991 .

[9]  Sin-Chung Chang.  The Method of Space-Time Conservation Element and Solution Element-A New Approach for Solving the Navier-Stokes and Euler Equations , 1995 .

[10]  R. F. Warming,et al.  Diagonalization and simultaneous symmetrization of the gas-dynamic matrices , 1975 .

[11]  Velocity-Stress Equations for Waves in Solids of Hexagonal Symmetry Solved by the Space-Time CESE Method , 2011 .

[12]  M. V. Dyke,et al.  An Album of Fluid Motion , 1982 .

[13]  Pascal Omnes,et al.  A finite volume method for the approximation of Maxwell's equations in two space dimensions on arbitrary meshes , 2008, J. Comput. Phys..

[14]  D. Mavriplis.  UNSTRUCTURED GRID TECHNIQUES , 1997 .

[15]  Brian Foote,et al.  Designing Reusable Classes , 2001 .

[16]  Sin-Chung Chang,et al.  A space-time conservation element and solution element method for solving the two- and three-dimensional unsteady euler equations using quadrilateral and hexahedral meshes , 2002 .

[17]  Hans Petter Langtangen.  A Case Study in High-Performance Mixed-Language Programming , 2006, PARA.

[18]  Michael Dumbser,et al.  An arbitrary high-order discontinuous Galerkin method for elastic waves on unstructured meshes - IV. Anisotropy , 2007 .

[19]  Yung-Yu Chen,et al.  SOLVCON: A Python-Based CFD Software Framework for Hybrid Parallelization , 2011 .

[20]  M F Sanner,et al.  Python: a programming language for software integration and development. , 1999, Journal of molecular graphics & modelling.

[21]  P. Raviart,et al.  Numerical Approximation of Hyperbolic Systems of Conservation Laws , 1996, Applied Mathematical Sciences.

[22]  Xing Cai,et al.  On the Efficiency of Python for High-Performance Computing: A Case Study Involving Stencil Updates for Partial Differential Equations , 2006, HPSC.

[23]  Michael Dumbser,et al.  Arbitrary high-order finite volume schemes for seismic wave propagation on unstructured meshes in 2D and 3D , 2007 .

[24]  Kwan-Liu Ma.  In situ visualization at extreme scale: challenges and opportunities. , 2009, IEEE computer graphics and applications.

[25]  Sin-Chung Chang.  On Space-Time Inversion Invariance and Its Relation to Non-Dissipatedness of a Cese Core Scheme , 2013 .

[26]  B. Auld,et al.  Acoustic fields and waves in solids , 1973 .

[27]  Sin-Chung Chang,et al.  New Developments in the Method of Space-Time Conservation Element and Solution Element-Applications to Two-Dimensional Time-Marching Problems , 1994 .

[28]  David McDaniel,et al.  Kestrel - A Fixed-Wing Virtual Aircraft Product of the CREATE Program , 2010 .

[29]  M. Musgrave,et al.  Crystal Acoustics: Introduction to the Study of Elastic Waves and Vibrations in Crystals , 1970 .

[30]  G. vanRossum.  Extending and embedding the Python interpreter , 1995 .

[31]  Yung-Yu Chen,et al.  Hyperbolicity of Velocity-Stress Equations for Waves in Anisotropic Elastic Solids , 2012 .

[32]  Doug Hellmann,et al.  The Python Standard Library by Example , 2011 .