Fast and Effective Lossy Compression Algorithms for Scientific Datasets

This paper focuses on developing effective and efficient algorithms for compressing scientific simulation data computed on structured and unstructured grids. A paradigm for lossy compression of this data is proposed in which the data computed on the grid is modeled as a graph, which gets decomposed into sets of vertices which satisfy a user defined error constraint e. Each set of vertices is replaced by a constant value with reconstruction error bounded by e. A comprehensive set of experiments is conducted by comparing these algorithms and other state-of-the-art scientific data compression methods. Over our benchmark suite, our methods obtained compression of 1% of the original size with average PSNR of 43.00 and 3% of the original size with average PSNR of 63.30. In addition, our schemes outperform other state-of-the-art lossy compression approaches and require on the average 25% of the space required by them for similar or better PSNR levels.

[1]  Martin Burtscher,et al.  Fast lossless compression of scientific floating-point data , 2006, Data Compression Conference (DCC'06).

[2]  Scott B. Baden,et al.  An Adaptive Sub-sampling Method for In-memory Compression of Scientific Data , 2009, 2009 Data Compression Conference.

[3]  Craig Gotsman,et al.  Spectral compression of mesh geometry , 2000, EuroCG.

[4]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  R. Coifman,et al.  Diffusion Wavelets , 2004 .

[6]  Eorge,et al.  Unstructured Graph Partitioning and Sparse Matrix Ordering System Version 2 . 0 , 1995 .

[7]  R. W. Hamming State of the art in scientific computing , 1963, AFIPS '63 (Spring).

[8]  Chuck Baldwin,et al.  Multi-resolution modeling of large scale scientific simulation data , 2003, CIKM '03.

[9]  Scott B. Baden,et al.  A Method of Adaptive Coarsening for Compressing Scientific Datasets , 2006, PARA.

[10]  Shigeru Muraki,et al.  Volume data and wavelet transforms , 1993, IEEE Computer Graphics and Applications.

[11]  Arthur D. Szlam,et al.  Diffusion wavelet packets , 2006 .

[12]  Peter Fritzson,et al.  Lossless compression of high-volume numerical data from simulations , 2000, Proceedings DCC 2000. Data Compression Conference.

[13]  R.A.F. Belfor,et al.  Spatially adaptive subsampling of image sequences , 1994, IEEE Trans. Image Process..

[14]  Yuefan Deng,et al.  Applied Parallel Computing , 2012 .

[15]  Shigeru Muraki,et al.  Approximation and rendering of volume data using wavelet transforms , 1992, Proceedings Visualization '92.