A multi-level parallel solver for rarefied gas flows in porous media

A high-performance gas kinetic solver using multi-level parallelization is developed to enable pore-scale simulations of rarefied flows in porous media. The Boltzmann model equation is solved by the discrete velocity method with an iterative scheme. The multi-level MPI/OpenMP parallelization is implemented with the aim to efficiently utilise the computational resources to allow direct simulation of rarefied gas flows in porous media based on digital rock images for the first time. The multi-level parallel approach is analyzed in details confirming its better performance than the commonly-used MPI processing alone for an iterative scheme. With high communication efficiency and appropriate load balancing among CPU processes, parallel efficiency of 94% is achieved for 1536 cores in the 2D simulations, and 81% for 12288 cores in the 3D simulations. While decomposition in the spatial space does not affect the simulation results, one additional benefit of this approach is that the number of subdomains can be kept minimal to avoid deterioration of the convergence rate of the iteration process. This multi-level parallel approach can be readily extended to solve other Boltzmann model equations.

[1]  Irina Graur,et al.  Rarefied gas flow through a zigzag channel , 2012 .

[2]  J. Broadwell,et al.  Study of rarefied shear flow by the discrete velocity method , 1964, Journal of Fluid Mechanics.

[3]  Felix Sharipov,et al.  Data on the Velocity Slip and Temperature Jump on a Gas-Solid Interface , 2011 .

[4]  Raphaël Loubère,et al.  Towards an ultra efficient kinetic scheme. Part I: Basics on the BGK equation , 2012, J. Comput. Phys..

[5]  J. Maxwell,et al.  On Stresses in Rarified Gases Arising from Inequalities of Temperature , 2022 .

[6]  V. V. Aristov,et al.  Unified solver for rarefied and continuum flows with adaptive mesh and algorithm refinement , 2007, J. Comput. Phys..

[7]  Juan-Chen Huang,et al.  Rarefied Flow Computations Using Nonlinear Model Boltzmann Equations , 1995 .

[8]  I. Lunati,et al.  A dual-tube model for gas dynamics in fractured nanoporous shale formations , 2014, Journal of Fluid Mechanics.

[9]  Songze Chen,et al.  dugksFoam: An open source OpenFOAM solver for the Boltzmann model equation , 2017, Comput. Phys. Commun..

[10]  Y. Qian,et al.  Lattice BGK Models for Navier-Stokes Equation , 1992 .

[11]  Joyce A. Dever,et al.  30th Aerospace Sciences Meeting and Exhibit , 1992 .

[12]  Farzam Javadpour,et al.  Gas flow in ultra-tight shale strata , 2012, Journal of Fluid Mechanics.

[13]  Yonghao Zhang,et al.  Gauss-Hermite quadratures and accuracy of lattice Boltzmann models for nonequilibrium gas flows. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[14]  E. M. Shakhov.  Generalization of the Krook kinetic relaxation equation , 1968 .

[15]  P. Bhatnagar,et al.  A Model for Collision Processes in Gases. I. Small Amplitude Processes in Charged and Neutral One-Component Systems , 1954 .

[16]  B. M. Fulk.  MATH , 1992 .

[17]  Josef Granwehr,et al.  Multiphase imaging of gas flow in a nanoporous material using remote-detection NMR , 2005, Nature materials.

[18]  Ao-Ping Peng,et al.  Rarefied gas flow simulations using high-order gas-kinetic unified algorithms for Boltzmann model equations , 2015 .

[19]  A. Acrivos,et al.  Slow flow through a periodic array of spheres , 1982 .

[20]  Livio Gibelli,et al.  Solving model kinetic equations on GPUs , 2011 .

[21]  Felix Sharipov,et al.  Data on Internal Rarefied Gas Flows , 1998 .

[22]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[23]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[24]  Yonghao Zhang,et al.  Accuracy analysis of high-order lattice Boltzmann models for rarefied gas flows , 2009, J. Comput. Phys..

[25]  Dimitris Valougeorgis,et al.  Acceleration Schemes of the Discrete Velocity Method: Gaseous Flows in Rectangular Microchannels , 2003, SIAM J. Sci. Comput..

[26]  Yonghao Zhang,et al.  Lattice ellipsoidal statistical BGK model for thermal non-equilibrium flows , 2012, Journal of Fluid Mechanics.

[27]  Thomas Scanlon,et al.  An open source, parallel DSMC code for rarefied gas flows in arbitrary geometries , 2010 .

[28]  Zhaoli Guo,et al.  A comparative study of discrete velocity methods for low-speed rarefied gas flows , 2018 .

[29]  M. Blunt,et al.  Pore-scale imaging and modelling , 2013 .

[30]  Michael Dumbser,et al.  Construction and comparison of parallel implicit kinetic solvers in three spatial dimensions , 2014, J. Comput. Phys..

[31]  Yonghao Zhang,et al.  On the apparent permeability of porous media in rarefied gas flows , 2017, Journal of Fluid Mechanics.

[32]  WALTER GAUTSCHI.  Algorithm 726: ORTHPOL–a package of routines for generating orthogonal polynomials and Gauss-type quadrature rules , 1994, TOMS.

[33]  Janet Elizabeth Hope.  Open Source , 2017, Encyclopedia of GIS.

[34]  François Pellegrini,et al.  PT-Scotch: A tool for efficient parallel graph ordering , 2008, Parallel Comput..

[35]  G. Bird.  Molecular Gas Dynamics and the Direct Simulation of Gas Flows , 1994 .

[36]  Qiang Wang,et al.  Natural gas from shale formation – The evolution, evidences and challenges of shale gas revolution in United States , 2014 .

[37]  Li Chen,et al.  The lattice Boltzmann method for isothermal micro-gaseous flow and its application in shale gas flow: a review , 2016 .

[38]  Zhaoli Guo,et al.  Discrete unified gas kinetic scheme for all Knudsen number flows: low-speed isothermal case. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[39]  Heinz Pitsch,et al.  Accuracy of higher-order lattice Boltzmann methods for microscale flows with finite Knudsen numbers , 2007, J. Comput. Phys..

[40]  Rainald Loehner,et al.  IMPLEMENTATION OF UNSTRUCTURED GRID GMRES+LU-SGS METHOD ON SHARED-MEMORY, CACHE-BASED PARALLEL COMPUTERS , 2000 .

[41]  Vladimir Titarev,et al.  Efficient Deterministic Modelling of Three-Dimensional Rarefied Gas Flows , 2012 .

[42]  Yu. Yu. Kloss,et al.  Programming and modelling environment for studies of gas flows in micro- and nanostructures based on solving the Boltzmann equation , 2008 .

[43]  Kun Xu,et al.  A unified gas-kinetic scheme for continuum and rarefied flows IV: Full Boltzmann and model equations , 2011, J. Comput. Phys..

[44]  L. Mieussens.  Discrete-Velocity Models and Numerical Schemes for the Boltzmann-BGK Equation in Plane and Axisymmetric Geometries , 2000 .

[45]  Georg Hager,et al.  Hybrid MPI/OpenMP Parallel Programming on Clusters of Multi-Core SMP Nodes , 2009, 2009 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing.

[46]  Raphaël Loubère,et al.  Towards an ultra efficient kinetic scheme. Part III: High-performance-computing , 2015, J. Comput. Phys..

[47]  Chengwen Zhong,et al.  Implicit unified gas-kinetic scheme for steady state solutions in all flow regimes , 2016, J. Comput. Phys..

[48]  Shi Jin,et al.  A class of asymptotic-preserving schemes for kinetic equations and related problems with stiff sources , 2009, J. Comput. Phys..

[49]  A. Ladd,et al.  Moderate-Reynolds-number flows in ordered and random arrays of spheres , 2001, Journal of Fluid Mechanics.

[50]  Duncan A. Lockerby,et al.  On the modelling of isothermal gas flows at the microscale , 2008, Journal of Fluid Mechanics.

[51]  Yonghao Zhang,et al.  Deterministic numerical solutions of the Boltzmann equation using the fast spectral method , 2013, J. Comput. Phys..

[52]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[53]  Livio Gibelli,et al.  Solving the Boltzmann equation on GPUs , 2011, Comput. Phys. Commun..

[54]  C. Baranger,et al.  Locally refined discrete velocity grids for stationary rarefied flow simulations , 2013, J. Comput. Phys..

[55]  Lowell H. Holway,et al.  New Statistical Models for Kinetic Theory: Methods of Construction , 1966 .

[56]  Sergei Utyuzhnikov,et al.  OpenMP + MPI parallel implementation of a numerical method for solving a kinetic equation , 2016 .

[57]  Lorenzo Pareschi,et al.  A Fourier spectral method for homogeneous boltzmann equations , 1996 .

[58]  Zhi-Hui Li,et al.  Gas-kinetic numerical studies of three-dimensional complex flows on spacecraft re-entry , 2009, J. Comput. Phys..