WaveRange: wavelet-based data compression for three-dimensional numerical simulations on regular grids

Abstract A wavelet-based method for compression of three-dimensional simulation data is presented and its software framework is described. It uses wavelet decomposition and subsequent range coding with quantization suitable for floating-point data. The effectiveness of this method is demonstrated by applying it to example numerical tests, ranging from idealized configurations to realistic global-scale simulations. The novelty of this study is in its focus on assessing the impact of compression on post-processing and restart of numerical simulations. Graphical abstract

[1]  G. Nigel Martin,et al.  * Range encoding: an algorithm for removing redundancy from a digitised message , 1979 .

[2]  Marc Antonini,et al.  HexaShrink, an exact scalable framework for hexahedral meshes with attributes and discontinuities: multiresolution rendering and storage of geoscience models , 2019, ArXiv.

[3]  O. Stelzer.  IND , 2020, Catalysis from A to Z.

[4]  Christopher M. Brislawn,et al.  Wavelet transform-vector quantization compression of supercomputer ocean models , 1993, [Proceedings] DCC `93: Data Compression Conference.

[5]  Ericka Stricklin-Parker,et al.  Ann , 2005 .

[6]  Masanori Kameyama,et al.  A 15.2 TFlops Simulation of Geodynamo on the Earth Simulator , 2004, Proceedings of the ACM/IEEE SC2004 Conference.

[7]  James P. Ahrens,et al.  Revisiting wavelet compression for large-scale climate data using JPEG 2000 and ensuring data precision , 2011, 2011 IEEE Symposium on Large Data Analysis and Visualization.

[8]  P. Holmes,et al.  The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows , 1993 .

[9]  Kazuhiro Nakahashi,et al.  Wavelet‐based data compression for flow simulation on block‐structured Cartesian mesh , 2013 .

[10]  Dong-Ho Lee,et al.  A study on CFD data compression using hybrid supercompact wavelets , 2003 .

[11]  S. Leerink,et al.  Gyrokinetic full-torus simulations of ohmic tokamak plasmas in circular limiter configuration , 2016, Comput. Phys. Commun..

[12]  Dimitri Komatitsch,et al.  The spectral-element method in seismology , 2013 .

[13]  Ryo Onishi,et al.  Tree-crown-resolving large-eddy simulation coupled with three-dimensional radiative transfer model , 2018 .

[14]  P. Schmid,et al.  Dynamic mode decomposition of numerical and experimental data , 2008, Journal of Fluid Mechanics.

[15]  D. Sasaki,et al.  Parallel implementation of large-scale CFD data compression toward aeroacoustic analysis , 2013 .

[16]  D. Sasaki,et al.  Large-Scale CFD Data Compression for Building-Cube Method Using Wavelet Transform , 2011 .

[17]  J. Lundquist,et al.  An Immersed Boundary Method for the Weather Research and Forecasting Model , 2014 .

[18]  Tage Røsten,et al.  Filter bank decomposition of seismic data with application to compression and denoising , 2000 .

[19]  Ryo Onishi,et al.  A Warm-Bin–Cold-Bulk Hybrid Cloud Microphysical Model* , 2012 .

[20]  A. Velazquez,et al.  Compression of aerodynamic databases using high-order singular value decomposition , 2010 .

[21]  John H. Day,et al.  Implementation of CCSDS Lossless Data Compression in HDF , 2002 .

[22]  Francesco De Simone,et al.  Evaluating lossy data compression on climate simulation data within a large ensemble , 2016, Geoscientific Model Development.

[23]  Ryo Onishi,et al.  Challenge toward the prediction of typhoon behaviour and down pour , 2013 .

[24]  Peter Deutsch,et al.  DEFLATE Compressed Data Format Specification version 1.3 , 1996, RFC.

[25]  장윤희,et al.  Y. , 2003, Industrial and Labor Relations Terms.

[26]  Keiko Takahashi,et al.  An efficient parallel simulation of interacting inertial particles in homogeneous isotropic turbulence , 2013, J. Comput. Phys..

[27]  Shintaro Kawahara,et al.  Global 7 km mesh nonhydrostatic Model Intercomparison Project for improving TYphoon forecast (TYMIP-G7): experimental design and preliminary results , 2016 .

[28]  B. Gudiksen,et al.  Non-thermal electrons from solar nanoflares , 2018, Astronomy & Astrophysics.

[29]  Hao Liu,et al.  Bumblebees minimize control challenges by combining active and passive modes in unsteady winds , 2016, Scientific Reports.

[30]  M. Hilbert,et al.  Big Data for Development: A Review of Promises and Challenges , 2016 .

[31]  Peter D. Düben,et al.  Improving Weather Forecast Skill through Reduced-Precision Data Assimilation , 2018 .

[32]  Kai Schneider,et al.  Reduced-Order Modelling of Turbulent Jets for Noise Control , 2009 .

[33]  P. Paolucci,et al.  The “Cubed Sphere” , 1996 .

[34]  Hank Childs,et al.  Data Reduction Techniques for Simulation, Visualization and Data Analysis , 2018, Comput. Graph. Forum.

[35]  O. Vasilyev,et al.  Wavelet Methods in Computational Fluid Dynamics , 2010 .

[36]  Michael W. Marcellin,et al.  JPEG2000 - image compression fundamentals, standards and practice , 2002, The Kluwer International Series in Engineering and Computer Science.

[37]  Chongke Bi,et al.  Parallel POD Compression of Time-Varying Big Datasets Using m-Swap on the K Computer , 2014, 2014 IEEE International Congress on Big Data.

[38]  Keiko Takahashi,et al.  Large-scale forcing with less communication in finite-difference simulations of stationary isotropic turbulence , 2011, J. Comput. Phys..

[39]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[40]  Teruyuki Nakajima,et al.  A k-distribution-based radiation code and its computational optimization for an atmospheric general circulation model , 2008 .

[41]  Renato Pajarola,et al.  State‐of‐the‐Art in Compressed GPU‐Based Direct Volume Rendering , 2014, Comput. Graph. Forum.

[42]  John P. Wilson.  Wavelet-based lossy compression of barotropic turbulence simulation data , 2002, Proceedings DCC 2002. Data Compression Conference.

[43]  Robert Latham,et al.  Compressing the Incompressible with ISABELA: In-situ Reduction of Spatio-temporal Data , 2011, Euro-Par.

[44]  L. Brandt,et al.  The effect of particle density in turbulent channel flow laden with finite size particles in semi-dilute conditions , 2015, 1511.01316.

[45]  Kenny Gruchalla,et al.  Evaluating the efficacy of wavelet configurations on turbulent-flow data , 2015, 2015 IEEE 5th Symposium on Large Data Analysis and Visualization (LDAV).

[46]  A. Kageyama,et al.  ``Yin-Yang grid'': An overset grid in spherical geometry , 2004, physics/0403123.

[47]  Claude E. Shannon,et al.  A mathematical theory of communication , 1948, MOCO.

[48]  Earl H. Dowell,et al.  Low-dimensional modelling of high-Reynolds-number shear flows incorporating constraints from the Navier–Stokes equation , 2013, Journal of Fluid Mechanics.

[49]  Sven Simon,et al.  Analyzing the Effect and Performance of Lossy Compression on Aeroacoustic Simulation of Gas Injector , 2017, Comput..

[50]  Yaliang Li,et al.  SCI , 2021, Proceedings of the 30th ACM International Conference on Information & Knowledge Management.

[51]  Andrew G. Glen,et al.  APPL , 2001 .

[52]  Tor A. Ramstad,et al.  Optimization of sub‐band coding method for seismic data compression , 2004 .

[53]  Franck Cappello,et al.  Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization , 2017, 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[54]  MON , 2020, Catalysis from A to Z.

[55]  M. Farge.  Wavelet Transforms and their Applications to Turbulence , 1992 .

[56]  Peter Lindstrom,et al.  Assessing the effects of data compression in simulations using physically motivated metrics , 2013, SC.

[57]  Shigeru Muraki,et al.  Volume data and wavelet transforms , 1993, IEEE Computer Graphics and Applications.

[58]  I. Daubechies,et al.  Factoring wavelet transforms into lifting steps , 1998 .