Compressive Neural Representations of Volumetric Scalar Fields

We present an approach for compressing volumetric scalar fields using implicit neural representations. Our approach represents a scalar field as a learned function, wherein a neural network maps a point in the domain to an output scalar value. By setting the number of weights of the neural network to be smaller than the input size, we achieve compressed representations of scalar fields, thus framing compression as a type of function approximation. Combined with carefully quantizing network weights, we show that this approach yields highly compact representations that outperform state‐of‐the‐art volume compression approaches. The conceptual simplicity of our approach enables a number of benefits, such as support for time‐varying scalar fields, optimizing to preserve spatial gradients, and random‐access field evaluation. We study the impact of network design choices on compression performance, highlighting how simple network architectures are effective for a broad range of volumes.

[1]  Peter Lindstrom,et al.  TTHRESH: Tensor Compression for Multidimensional Visual Data , 2018, IEEE Transactions on Visualization and Computer Graphics.

[2]  Yi Li,et al.  A public turbulence database cluster and applications to study Lagrangian evolution of velocity increments in turbulence , 2008, 0804.1703.

[3]  Hao Zhang,et al.  Learning Implicit Fields for Generative Shape Modeling , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Wolfgang Straßer,et al.  Interactive rendering of large volume data sets , 2002, IEEE Visualization, 2002. VIS 2002..

[5]  Song Han,et al.  Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding , 2015, ICLR.

[6]  Robert Latham,et al.  Compressing the Incompressible with ISABELA: In-situ Reduction of Spatio-temporal Data , 2011, Euro-Par.

[7]  Renato Pajarola,et al.  TAMRESH – Tensor Approximation Multiresolution Hierarchy for Interactive Volume Visualization , 2013, Comput. Graph. Forum.

[8]  Aaron Knoll,et al.  OSPRay - A CPU Ray Tracing Framework for Scientific Visualization , 2017, IEEE Transactions on Visualization and Computer Graphics.

[9]  Gordon Wetzstein,et al.  Implicit Neural Representations with Periodic Activation Functions , 2020, NeurIPS.

[10]  Shigeru Muraki,et al.  Volume data and wavelet transforms , 1993, IEEE Computer Graphics and Applications.

[11]  Insung Ihm,et al.  Wavelet‐Based 3D Compression Scheme for Interactive Visualization of Very Large Volume Data , 1999, Comput. Graph. Forum.

[12]  James P. Ahrens,et al.  Revisiting wavelet compression for large-scale climate data using JPEG 2000 and ensuring data precision , 2011, 2011 IEEE Symposium on Large Data Analysis and Visualization.

[13]  Ray W. Grout,et al.  Stability of Dissipation Elements: A Case Study in Combustion , 2014, Comput. Graph. Forum.

[14]  William Daughton,et al.  Formation of hard power laws in the energetic particle spectra resulting from relativistic magnetic reconnection. , 2014, Physical review letters.

[15]  Franck Cappello,et al.  Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization , 2017, 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[16]  Jens Schneider,et al.  Compression domain volume rendering , 2003, IEEE Visualization, 2003. VIS 2003..

[17]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[18]  Noah Snavely,et al.  DualSDF: Semantic Shape Manipulation Using a Two-Level Representation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Mukund Raj,et al.  InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations , 2019, IEEE Transactions on Visualization and Computer Graphics.

[20]  Andreas Geiger,et al.  Texture Fields: Learning Texture Representations in Function Space , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[21]  Valerio Pascucci,et al.  Efficient and Flexible Hierarchical Data Layouts for a Unified Encoding of Scalar Field Precision and Resolution , 2020, IEEE Transactions on Visualization and Computer Graphics.

[22]  Pratul P. Srinivasan,et al.  NeRF , 2020, ECCV.

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  Li Guo,et al.  SSR-VFD: Spatial Super-Resolution for Vector Field Data Analysis and Visualization , 2020, 2020 IEEE Pacific Visualization Symposium (PacificVis).

[25]  Markus Gross,et al.  A Fluid Flow Data Set for Machine Learning and its Application to Neural Flow Map Interpolation , 2020, IEEE Transactions on Visualization and Computer Graphics.

[26]  Rüdiger Westermann,et al.  Volumetric Isosurface Rendering with Deep Learning-Based Super-Resolution , 2019, IEEE Transactions on Visualization and Computer Graphics.

[27]  Enrico Gobbetti,et al.  Interactive spatio-temporal exploration of massive time-Varying rectilinear scalar volumes based on a variable bit-rate sparse representation over learned dictionaries , 2020, Comput. Graph..

[28]  Franck Cappello,et al.  Error-Controlled Lossy Compression Optimized for High Compression Ratios of Scientific Datasets , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[29]  Boon-Lock Yeo,et al.  Volume Rendering of DCT-Based Compressed 3D Scalar Data , 1995, IEEE Trans. Vis. Comput. Graph..

[30]  Jeremy Iverson,et al.  Fast and Effective Lossy Compression Algorithms for Scientific Datasets , 2012, Euro-Par.

[31]  Joshua A. Levine,et al.  A Generative Model for Volume Rendering , 2017, IEEE Transactions on Visualization and Computer Graphics.

[32]  Jun Han,et al.  SSR-TVD: Spatial Super-Resolution for Time-Varying Data Analysis and Visualization , 2020, IEEE Transactions on Visualization and Computer Graphics.

[33]  Renato Pajarola,et al.  Lossy volume compression using Tucker truncation and thresholding , 2016, The Visual Computer.

[34]  Sebastian Nowozin,et al.  Occupancy Networks: Learning 3D Reconstruction in Function Space , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Alec Jacobson,et al.  Overfit Neural Networks as a Compact Shape Representation , 2020, ArXiv.

[36]  Julien Tierny,et al.  Topologically Controlled Lossy Compression , 2018, 2018 IEEE Pacific Visualization Symposium (PacificVis).

[37]  Peter Lindstrom,et al.  Fixed-Rate Compressed Floating-Point Arrays , 2014, IEEE Transactions on Visualization and Computer Graphics.

[38]  Andrew W. Cook,et al.  The mixing transition in Rayleigh–Taylor instability , 2004, Journal of Fluid Mechanics.

[39]  Matthias Zwicker,et al.  SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Hao Zheng,et al.  V2V: A Deep Learning Approach to Variable-to-Variable Selection and Translation for Multivariate Time-Varying Data , 2020, IEEE Transactions on Visualization and Computer Graphics.

[41]  Jonathan T. Barron,et al.  Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains , 2020, NeurIPS.

[42]  Danny Z. Chen,et al.  Flow Field Reduction Via Reconstructing Vector Data From 3-D Streamlines Using Deep Learning , 2019, IEEE Computer Graphics and Applications.

[43]  Enrico Gobbetti,et al.  COVRA: A compression‐domain output‐sensitive volume rendering architecture based on a sparse representation of voxel blocks , 2012, Comput. Graph. Forum.

[44]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[45]  Iulian Grindeanu,et al.  Foundations of Multivariate Functional Approximation for Scientific Data , 2018, 2018 IEEE 8th Symposium on Large Data Analysis and Visualization (LDAV).

[46]  Franck Cappello,et al.  Fast Error-Bounded Lossy HPC Data Compression with SZ , 2016, 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[47]  Chaoli Wang,et al.  TSR-TVD: Temporal Super-Resolution for Time-Varying Data Analysis and Visualization , 2020, IEEE Transactions on Visualization and Computer Graphics.

[48]  Renato Pajarola,et al.  A Survey of Compressed GPU-Based Direct Volume Rendering , 2013, Eurographics.

[49]  Richard A. Newcombe,et al.  DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).