The Mixture Graph-A Data Structure for Compressing, Rendering, and Querying Segmentation Histograms

In this paper, we present a novel data structure, called the Mixture Graph. This data structure allows us to compress, render, and query segmentation histograms. Such histograms arise when building a mipmap of a volume containing segmentation IDs. Each voxel in the histogram mipmap contains a convex combination (mixture) of segmentation IDs. Each mixture represents the distribution of IDs in the respective voxel's children. Our method factorizes these mixtures into a series of linear interpolations between exactly two segmentation IDs. The result is represented as a directed acyclic graph (DAG) whose nodes are topologically ordered. Pruning replicate nodes in the tree followed by compression allows us to store the resulting data structure efficiently. During rendering, transfer functions are propagated from sources (leafs) through the DAG to allow for efficient, pre-filtered rendering at interactive frame rates. Assembly of histogram contributions across the footprint of a given volume allows us to efficiently query partial histograms, achieving up to 178× speed-up over naïve parallelized range queries. Additionally, we apply the Mixture Graph to compute correctly pre-filtered volume lighting and to interactively explore segments based on shape, geometry, and orientation using multi-dimensional transfer functions.

[1]  Eric L. Miller,et al.  Segmentation fusion for connectomics , 2011, 2011 International Conference on Computer Vision.

[2]  Allen Gersho,et al.  Vector quantization and signal compression , 1991, The Kluwer international series in engineering and computer science.

[3]  Kenneth A. Ross,et al.  Massively-Parallel Lossless Data Decompression , 2016, 2016 45th International Conference on Parallel Processing (ICPP).

[4]  Insung Ihm,et al.  Wavelet‐Based 3D Compression Scheme for Interactive Visualization of Very Large Volume Data , 1999, Comput. Graph. Forum.

[5]  Bertil Schmidt,et al.  Massively Parallel ANS Decoding on GPUs , 2019, ICPP.

[6]  Paolo Cignoni,et al.  Real Time, Accurate, Multi‐Featured Rendering of Bump Mapped Surfaces , 2000, Comput. Graph. Forum.

[7]  Dietmar Saupe,et al.  Rapid High Quality Compression of Volume Data for Visualization , 2001, Comput. Graph. Forum.

[8]  Markus Hadwiger,et al.  SparseLeap: Efficient Empty Space Skipping for Large-Scale Volume Rendering , 2018, IEEE Transactions on Visualization and Computer Graphics.

[9]  Kwan-Liu Ma,et al.  Transform Coding for Hardware-accelerated Volume Rendering , 2007, IEEE Transactions on Visualization and Computer Graphics.

[10]  Paul Ning,et al.  Vector quantization for volume rendering , 1992, VVS.

[11]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[12]  Markus Hadwiger,et al.  NeuroBlocks – Visual Tracking of Segmentation and Proofreading for Large Connectomics Projects , 2016, IEEE Transactions on Visualization and Computer Graphics.

[13]  Han-Wei Shen,et al.  Image and Distribution Based Volume Rendering for Large Data Sets , 2018, 2018 IEEE Pacific Visualization Symposium (PacificVis).

[14]  Dirk Schlimm,et al.  David Hilbert's Lectures on the Foundations of Arithmetic and Logic 1917-1933 , 2013 .

[15]  Heikki Lehväslaiho,et al.  Three‐dimensional immersive virtual reality for studying cellular compartments in 3D models from EM preparations of neural tissues , 2015, The Journal of comparative neurology.

[16]  Eduard Gröller,et al.  FiberScout: An Interactive Tool for Exploring and Analyzing Fiber Reinforced Polymers , 2014, 2014 IEEE Pacific Visualization Symposium.

[17]  Gregory D. Hager,et al.  VESICLE: Volumetric Evaluation of Synaptic Inferfaces using Computer Vision at Large Scale , 2014, BMVC.

[18]  I. Daubechies,et al.  Biorthogonal bases of compactly supported wavelets , 1992 .

[19]  Yao Zhang,et al.  Parallel lossless data compression on the GPU , 2012, 2012 Innovative Parallel Computing (InPar).

[20]  Markus Hadwiger,et al.  High-quality two-level volume rendering of segmented data sets on consumer graphics hardware , 2003, IEEE Visualization, 2003. VIS 2003..

[21]  Eduard Gröller,et al.  Two-Level Volume Rendering , 2001, IEEE Trans. Vis. Comput. Graph..

[22]  Insung Ihm,et al.  3D RGB image compression for interactive applications , 2001, TOGS.

[23]  PfisterHanspeter,et al.  Scalable and Interactive Segmentation and Visualization of Neural Processes in EM Datasets , 2009 .

[24]  Koji Nakano,et al.  Adaptive loss‐less data compression method optimized for GPU decompression , 2017, Concurr. Comput. Pract. Exp..

[25]  Rüdiger Westermann,et al.  Interactive Editing of GigaSample Terrain Fields , 2012, Comput. Graph. Forum.

[26]  Rüdiger Westermann,et al.  Visualization of big SPH simulations via compressed octree grids , 2013, 2013 IEEE International Conference on Big Data.

[27]  Markus Hadwiger,et al.  Culling for Extreme-Scale Segmentation Volumes: A Hybrid Deterministic and Probabilistic Approach , 2019, IEEE Transactions on Visualization and Computer Graphics.

[28]  Won-Ki Jeong,et al.  Interactive Histology of Large-Scale Biomedical Image Stacks , 2010, IEEE Transactions on Visualization and Computer Graphics.

[29]  Anders Ynnerman,et al.  Local Ambient Occlusion in Direct Volume Rendering , 2010, IEEE Transactions on Visualization and Computer Graphics.

[30]  J. Lichtman,et al.  Imaging a 1 mm 3 Volume of Rat Cortex Using a MultiBeam SEM , 2016, Microscopy and Microanalysis.

[31]  Wolfgang Straßer,et al.  Texram: a smart memory for texturing , 1996, IEEE Computer Graphics and Applications.

[32]  William A. Pearlman,et al.  A new, fast, and efficient image codec based on set partitioning in hierarchical trees , 1996, IEEE Trans. Circuits Syst. Video Technol..

[33]  Glen G. Langdon,et al.  Arithmetic Coding , 1979 .

[34]  Markus Hadwiger,et al.  ConnectomeExplorer: Query-Guided Visual Analysis of Large Volumetric Neuroscience Data , 2013, IEEE Transactions on Visualization and Computer Graphics.

[35]  Markus Hadwiger,et al.  Exploring the Connectome: Petascale Volume Visualization of Microscopy Data Streams , 2013, IEEE Computer Graphics and Applications.

[36]  Markus Hadwiger,et al.  Ssecrett and NeuroTrace: Interactive Visualization and Analysis Tools for Large-Scale Neuroscience Data Sets , 2010, IEEE Computer Graphics and Applications.

[37]  Markus Hadwiger,et al.  Interactive Volume Exploration of Petascale Microscopy Data Streams Using a Visualization-Driven Virtual Memory Approach , 2012, IEEE Transactions on Visualization and Computer Graphics.

[38]  Peter Stucki,et al.  Funfamentals of 3D Halftoning , 1998, EP.

[39]  Michael Elad,et al.  Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation , 2010, IEEE Transactions on Signal Processing.

[40]  Victor S. Lempitsky,et al.  Surface extraction from binary volumes with higher-order smoothness , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Amelio Vázquez Reina,et al.  Large-Scale Automatic Reconstruction of Neuronal Processes from Electron Microscopy Images , 2013, Medical Image Anal..

[42]  Han-Wei Shen,et al.  Statistical visualization and analysis of large data using a value-based spatial distribution , 2017, 2017 IEEE Pacific Visualization Symposium (PacificVis).

[43]  Lance Williams,et al.  Pyramidal parametrics , 1983, SIGGRAPH.

[44]  William R. Gray Roncal,et al.  Saturated Reconstruction of a Volume of Neocortex , 2015, Cell.

[45]  Dinesh Manocha,et al.  Appearance-preserving simplification , 1998, SIGGRAPH.

[46]  Leif Kobbelt,et al.  Two‐Colored Pixels , 2010, Comput. Graph. Forum.

[47]  Jens H. Krüger,et al.  State of the Art in Transfer Functions for Direct Volume Rendering , 2016, Comput. Graph. Forum.

[48]  Shigeru Muraki,et al.  Volume data and wavelet transforms , 1993, IEEE Computer Graphics and Applications.

[49]  I. Daubechies,et al.  Factoring wavelet transforms into lifting steps , 1998 .

[50]  Martin Kraus,et al.  GPU-Based Euclidean Distance Transforms and Their Application to Volume Rendering , 2009, VISIGRAPP.

[51]  Jens Schneider,et al.  Compression domain volume rendering , 2003, IEEE Visualization, 2003. VIS 2003..

[52]  Markus Hadwiger,et al.  NeuroLines: A Subway Map Metaphor for Visualizing Nanoscale Neuronal Connectivity , 2014, IEEE Transactions on Visualization and Computer Graphics.

[53]  Johanna Beyer,et al.  Design and Evaluation of Interactive Proofreading Tools for Connectomics , 2014, IEEE Transactions on Visualization and Computer Graphics.

[54]  Bertil Schmidt,et al.  Massively Parallel Huffman Decoding on GPUs , 2018, ICPP.