On the accuracy and performance of the lattice Boltzmann method with 64-bit, 32-bit and novel 16-bit number formats

Fluid dynamics simulations with the lattice Boltzmann method (LBM) are very memory intensive. Alongside reduction in memory footprint, significant performance benefits can be achieved by using FP32 (single) precision compared to FP64 (double) precision, especially on GPUs. Here we evaluate the possibility to use even FP16 and posit16 (half) precision for storing fluid populations, while still carrying arithmetic operations in FP32. For this, we first show that the commonly occurring number range in the LBM is a lot smaller than the FP16 number range. Based on this observation, we develop customized 16-bit formats-based on a modified IEEE-754 and on a modified posit standard-that are specifically tailored to the needs of the LBM. We then carry out an in-depth characterization of LBM accuracy for six different test systems with increasing complexity: Poiseuille flow, Taylor-Green vortices, Karman vortex streets, lid-driven cavity, a microcapsule in shear flow (utilizing the immersed-boundary method), and, finally, the impact of a raindrop (based on a volume-of-fluid approach). We find that the difference in accuracy between FP64 and FP32 is negligible in almost all cases, and that for a large number of cases even 16-bit is sufficient. Finally, we provide a detailed performance analysis of all precision levels on a large number of hardware microarchitectures and show that significant speedup is achieved with mixed FP32/16-bit.

[1]  M. Frank,et al.  Implicit propagation of directly addressed grids in lattice Boltzmann methods , 2023, Concurr. Comput. Pract. Exp..

[2]  Moritz Lehmann Esoteric Pull and Esoteric Push: Two Simple In-Place Streaming Schemes for the Lattice Boltzmann Method on GPUs , 2022, Comput..

[3]  S. Gekle,et al.  Ejection of marine microplastics by raindrops: a computational and experimental study , 2021, Microplastics and Nanoplastics.

[4]  Ivo Petráš,et al.  Cross-Platform GPU-Based Implementation of Lattice Boltzmann Method Solver Using ArrayFire Library , 2021, Mathematics.

[5]  C. Bogner,et al.  Tracing the horizontal transport of microplastics on rough surfaces , 2021, Microplastics and Nanoplastics.

[6]  M. Porfiri,et al.  Extreme flow simulations reveal skeletal adaptations of deep-sea sponges , 2021, Nature.

[7]  Peter Dueben,et al.  Compressing atmospheric data into its real information content , 2021, Nature Computational Science.

[8]  A. Koelmans An inclusive, interdisciplinary forum for worldwide efforts to solve the plastics pollution challenge , 2021, Microplastics and Nanoplastics.

[9]  J. Harting,et al.  Lattice Boltzmann simulations of stochastic thin film dewetting. , 2020, Physical review. E.

[10]  Martin Bauer,et al.  Highly efficient lattice Boltzmann multiphase simulations of immiscible fluids at high-density ratios on CPUs and GPUs through code generation , 2020, Int. J. High Perform. Comput. Appl..

[11]  Jonathan B. Jeppener-Haltenhoff,et al.  OpenLB Release 1.4: Open Source Lattice Boltzmann Code , 2020 .

[12]  Bastien Chopard,et al.  Palabos-npFEM: Software for the Simulation of Cellular Blood Flow (Digital Blood) , 2020, Journal of Open Research Software.

[13]  Nicholas J. Higham,et al.  Mixed-precision iterative refinement using tensor cores on GPUs to accelerate solution of linear systems , 2020, Proceedings of the Royal Society A.

[14]  F. Auricchio,et al.  Basis of the Lattice Boltzmann Method for Additive Manufacturing , 2020, Archives of Computational Methods in Engineering.

[15]  Erik W Draeger,et al.  Multi-GPU immersed boundary method hemodynamics simulations , 2020, J. Comput. Sci..

[16]  Stephan Gekle,et al.  Analytic Solution to the Piecewise Linear Interface Construction Problem and its Application in Curvature Calculation for Volume-of-Fluid Simulation Codes , 2020, Comput..

[17]  J. Latt,et al.  Efficient supersonic flow simulations using lattice Boltzmann methods based on numerical equilibria , 2020, Philosophical Transactions of the Royal Society A.

[18]  Fabian Klemens,et al.  OpenLB - Open source lattice Boltzmann code , 2020, Comput. Math. Appl..

[19]  Andreas Lintermann,et al.  Lattice–Boltzmann simulations for complex geometries on high-performance computers , 2020, CEAS Aeronautical Journal.

[20]  Orestis Malaspinas,et al.  Palabos: Parallel Lattice Boltzmann Solver , 2020, Comput. Math. Appl..

[21]  Massimo Bernaschi,et al.  LBsoft: A parallel open-source software for simulation of colloidal systems , 2020, Comput. Phys. Commun..

[22]  H. Yoshida,et al.  Locally mesh-refined lattice Boltzmann method for fuel debris air cooling analysis on GPU supercomputer , 2020 .

[23]  J. Harting,et al.  Lattice Boltzmann method for thin-liquid-film hydrodynamics. , 2018, Physical review. E.

[24]  Matthew Chantry,et al.  Accelerating High-Resolution Weather Models with Deep-Learning Hardware , 2019, PASC.

[25]  Jonas Latt,et al.  Toward Full GPU Implementation of Fluid-Structure Interaction , 2019, 2019 18th International Symposium on Parallel and Distributed Computing (ISPDC).

[26]  Enrique S. Quintana-Ortí,et al.  Toward a modular precision ecosystem for high-performance computing , 2019, Int. J. High Perform. Comput. Appl..

[27]  Jonas Latt,et al.  Efficient LBM on GPUs for dense moving objects using immersed boundary condition , 2019, ArXiv.

[28]  Bastien Chopard,et al.  Bridging the computational gap between mesoscopic and continuum modeling of red blood cells for fully resolved blood flow , 2019, J. Comput. Phys..

[29]  Massimo Bernaschi,et al.  Towards Exascale Lattice Boltzmann computing , 2019, Computers & Fluids.

[30]  Jean-Michel Muller,et al.  Posits: the good, the bad and the ugly , 2019, CoNGA'19.

[31]  Gudrun Thäter,et al.  Auto-vectorization friendly parallel lattice Boltzmann streaming scheme for direct addressing , 2019, Computers & Fluids.

[32]  J. Harting,et al.  SWALBE: A lattice Boltzmann solver of the shallow water equations for thin liquid film flows , 2018, 1810.07642.

[33]  Hartwig Anzt,et al.  A Modular Precision Format for Decoupling Arithmetic Format and Storage Format , 2018, Euro-Par Workshops.

[34]  Seyong Lee,et al.  GPU Data Access on Complex Geometries for D3Q19 Lattice Boltzmann Method , 2018, 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[35]  J. Harting,et al.  Mesoscopic electrohydrodynamic simulations of binary colloidal suspensions. , 2017, The Journal of chemical physics.

[36]  Benoît Dupont de Dinechin,et al.  Improving 3D lattice boltzmann method stencil with asynchronous transfers on many-core processors , 2017, 2017 IEEE 36th International Performance Computing and Communications Conference (IPCCC).

[37]  Hans-Joachim Bungartz,et al.  A Holistic Scalable Implementation Approach of the Lattice Boltzmann Method for CPU/GPU Heterogeneous Clusters , 2017, Comput..

[38]  Peter V. Coveney,et al.  LB3D: A parallel implementation of the Lattice-Boltzmann method for simulation of interacting amphiphilic fluids , 2017, Comput. Phys. Commun..

[39]  A. Purqon,et al.  Accuracy and Numerical Stabilty Analysis of Lattice Boltzmann Method with Multiple Relaxation Time for Incompressible Flows , 2017 .

[40]  John L. Gustafson,et al.  Beating Floating Point at its Own Game: Posit Arithmetic , 2017, Supercomput. Front. Innov..

[41]  S. Gekle,et al.  Theory and algorithms to compute Helfrich bending forces: a review , 2017, Journal of physics. Condensed matter : an Institute of Physics journal.

[42]  G. Boroni,et al.  FULL GPU Implementation of Lattice-Boltzmann Methods with Immersed Boundary Conditions for Fast Fluid Simulations , 2017 .

[43]  Martin Geier,et al.  Esoteric Twist: An Efficient in-Place Streaming Algorithmus for the Lattice Boltzmann Method on Massively Parallel Hardware , 2017, Comput..

[44]  Alistair J. Revell,et al.  Parallelisation of an interactive lattice-Boltzmann method on an Android-powered mobile device , 2017, Adv. Eng. Softw..

[45]  Stephan Gekle,et al.  On the bending algorithms for soft objects in flows , 2016, Comput. Phys. Commun..

[46]  Raffaele Tripiccione,et al.  Massively parallel lattice-Boltzmann codes on large GPU clusters , 2016, Parallel Comput..

[47]  Philipp Neumann,et al.  Lattice Boltzmann Flow Simulation on Android Devices for Interactive Mobile-Based Learning , 2016, Euro-Par Workshops.

[48]  Edo S. Boek,et al.  Enhancing Computational Precision for Lattice Boltzmann Schemes in Porous Media Flows , 2016, Comput..

[49]  Dominique Barthès-Biesel,et al.  Motion and Deformation of Elastic Capsules and Vesicles in Flow , 2016 .

[50]  Christophe Renaud,et al.  Accelerating Physical Simulations from a Multicomponent Lattice Boltzmann Method on a Single-Node Multi-GPU Architecture , 2015, 2015 10th International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC).

[51]  Raffaele Tripiccione,et al.  Optimizing communications in multi-GPU Lattice Boltzmann simulations , 2015, 2015 International Conference on High Performance Computing & Simulation (HPCS).

[52]  Thomas Rung,et al.  Validation of the GPU-Accelerated CFD Solver ELBE for Free Surface Flow Problems in Civil and Environmental Engineering , 2015, Comput..

[53]  Orestis Malaspinas,et al.  Parallel performance of an IB-LBM suspension simulation framework , 2015, J. Comput. Sci..

[54]  Li-Min Huang,et al.  Scalable multi-relaxation-time lattice Boltzmann simulations on multi-GPU cluster , 2015 .

[55]  D. Wolf,et al.  Recent advances in the simulation of particle-laden flows , 2014, The European Physical Journal Special Topics.

[56]  C. J. Noakes,et al.  Optimized implementation of the Lattice Boltzmann Method on a graphics processing unit towards real-time fluid simulation , 2014, Comput. Math. Appl..

[57]  J. Harting,et al.  Interplay of inertia and deformability on rheological properties of a suspension of capsules , 2013, Journal of Fluid Mechanics.

[58]  Alistair J. Revell,et al.  Memory transfer optimization for a lattice Boltzmann solver on Kepler architecture nVidia GPUs , 2013, Comput. Phys. Commun..

[59]  W. Gu,et al.  Performance Analysis and Optimization of PalaBos on Petascale Sunway BlueLight MPP Supercomputer , 2013, ParCo 2013.

[60]  T. Neckel,et al.  A dynamic mesh refinement technique for Lattice Boltzmann simulations on octree-like grids , 2013 .

[61]  B. Kaoui,et al.  Numerical simulations of complex fluid-fluid interface dynamics , 2012, 1208.2539.

[62]  Marcelo J. Vénere,et al.  A Lattice-Boltzmann solver for 3D fluid simulation on GPU , 2012, Simul. Model. Pract. Theory.

[63]  Ge Wei,et al.  Efficient parallel implementation of the lattice Boltzmann method on large clusters of graphic processing units , 2012 .

[64]  Gerhard Wellein,et al.  Performance engineering for the Lattice Boltzmann method on GPGPUs: Architectural requirements and performance results , 2011, ArXiv.

[65]  Gerhard Wellein,et al.  Comparison of different propagation steps for lattice Boltzmann methods , 2011, Comput. Math. Appl..

[66]  Bernard Tourancheau,et al.  A new approach to the lattice Boltzmann method for graphics processing units , 2011, Comput. Math. Appl..

[67]  Gerhard Wellein,et al.  Performance analysis and optimization strategies for a D3Q19 lattice Boltzmann kernel on nVIDIA GPUs using CUDA , 2011, Adv. Eng. Softw..

[68]  Ulrich Rüde,et al.  A flexible Patch-based lattice Boltzmann parallelization approach for heterogeneous GPU-CPU clusters , 2010, Parallel Comput..

[69]  Bernard Tourancheau,et al.  Global Memory Access Modelling for Efficient Implementation of the Lattice Boltzmann Method on Graphics Processing Units , 2010, VECPAR.

[70]  A. Salsac,et al.  Flow-Induced Deformation of Artificial Capsules , 2010 .

[71]  Christian Obrecht,et al.  LBM based flow simulation using GPU computing processor , 2010, Comput. Math. Appl..

[72]  Peter Bailey,et al.  Accelerating geoscience and engineering system simulations on graphics hardware , 2009, Comput. Geosci..

[73]  Peter Bailey,et al.  Accelerating Lattice Boltzmann Fluid Flow Simulations Using Graphics Processors , 2009, 2009 International Conference on Parallel Processing.

[74]  Norberto Fueyo,et al.  Analysis of open boundary effects in unsteady lattice Boltzmann simulations , 2009, Comput. Math. Appl..

[75]  Alireza Entezari,et al.  The Lattice-Boltzmann Method on Optimal Sampling Lattices , 2009, IEEE Transactions on Visualization and Computer Graphics.

[76]  Manfred Krafczyk,et al.  TeraFLOP computing on a desktop PC with GPUs for 3D CFD , 2008 .

[77]  J. Dongarra,et al.  Exploiting the Performance of 32 bit Floating Point Arithmetic in Obtaining 64 bit Accuracy (Revisiting Iterative Refinement for Linear Systems) , 2006, ACM/IEEE SC 2006 Conference (SC'06).

[78]  Gerhard Wellein,et al.  On the single processor performance of simple lattice Boltzmann kernels , 2006 .

[79]  Hans-Jörg Limbach,et al.  ESPResSo - an extensible simulation package for research on soft matter systems , 2006, Comput. Phys. Commun..

[80]  B. Shi,et al.  Discrete lattice effects on the forcing term in the lattice Boltzmann method. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[81]  D. d'Humières,et al.  Multiple–relaxation–time lattice Boltzmann models in three dimensions , 2002, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[82]  Jaw-Yen Yang,et al.  Implicit Weighted ENO Schemes for the Three-Dimensional Incompressible Navier-Stokes Equations , 1998 .

[83]  L. Luo,et al.  Lattice Boltzmann Model for the Incompressible Navier–Stokes Equation , 1997 .

[84]  Louis A. Povinelli,et al.  Large-scale computation of incompressible viscous flow by least-squares finite element method , 1994 .

[85]  A. Ladd Numerical simulations of particulate suspensions via a discretized Boltzmann equation. Part 1. Theoretical foundation , 1993, Journal of Fluid Mechanics.

[86]  Skordos,et al.  Initial and boundary conditions for the lattice Boltzmann method. , 1993, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[87]  R. Benzi,et al.  The lattice Boltzmann equation: theory and applications , 1992 .

[88]  David Goldberg,et al.  What every computer scientist should know about floating-point arithmetic , 1991, CSUR.

[89]  U. Ghia,et al.  High-Re solutions for incompressible flow using the Navier-Stokes equations and a multigrid method , 1982 .

[90]  G. Batchelor,et al.  An Introduction to Fluid Dynamics , 1968 .

[91]  T. G. Cowling,et al.  The mathematical theory of non-uniform gases : an account of the kinetic theory of viscosity, thermal conduction, and diffusion in gases , 1954 .

[92]  G. Taylor,et al.  Mechanism of the production of small eddies from large ones , 1937 .

[93]  Moritz Lehmann High Performance Free Surface LBM on GPUs , 2021 .

[94]  Zhangrong Qin,et al.  An Efficient Graphics Processing Unit Scheme for Complex Geometry Simulations Using the Lattice Boltzmann Method , 2020, IEEE Access.

[95]  Erlend Magnus Viggen,et al.  The Lattice Boltzmann Method , 2017 .

[96]  Nhat-Phuong Tran,et al.  Performance Optimization of 3D Lattice Boltzmann Flow Solver on a GPU , 2017, Sci. Program..

[97]  Markus Wittmann,et al.  Hardware-effiziente, hochparallele Implementierungen von Lattice-Boltzmann-Verfahren für komplexe Geometrien , 2016 .

[98]  Bernard Tourancheau,et al.  Multi-GPU implementation of the lattice Boltzmann method , 2013, Comput. Math. Appl..

[99]  Miloš Racković,et al.  Implementation of the Lattice Boltzmann Method on Heterogeneous Hardware and Platforms using OpenCL , 2012 .

[100]  Takayuki Aoki,et al.  Multi-GPU performance of incompressible flow computation by lattice Boltzmann method on GPU cluster , 2011, Parallel Comput..

[101]  Hans-Joachim Bungartz,et al.  International Conference on Computational Science , ICCS 2011 Free-Surface Lattice-Boltzmann Simulation on Many-Core Architectures , 2011 .

[102]  Mathias J. Krause,et al.  Fluid Flow Simulation and Optimisation with Lattice Boltzmann Methods on High Performance Computers - Application to the Human Respiratory System , 2010 .

[103]  Stefan Turek,et al.  Lattice-Boltzmann Simulation of the Shallow-Water Equations with Fluid-Structure Interaction on Multi- and Manycore Processors , 2010, Facing the Multicore-Challenge.

[104]  Anne C. Elster,et al.  Porous Rock Simulations and Lattice Boltzmann on GPUs , 2009, PARCO.

[105]  Mathias J. Krause,et al.  OpenLB : towards an efficient parallel open source library for lattice Boltzmann fluid flow simulations , 2008 .

[106]  Samuel Williams,et al.  Roofline: An Insightful Visual Performance Model for Floating-Point Programs and Multicore Architectures , 2008 .

[107]  Gerhard Wellein,et al.  Towards Optimal Performance for Lattice Boltzmann Applications on Terascale Computers , 2006 .

[108]  Michael Griebel,et al.  Meshfree Methods for Partial Differential Equations IV , 2005 .

[109]  T. Kármán,et al.  Ueber den Mechanismus des Widerstandes, den ein bewegter Körper in einer Flüssigkeit erfährt , 1911 .

[110]  Introduction to the immersed boundary method 1 Motivation 1 . 1 Hydrodynamics and boundary conditions , 2022 .