Scalable isosurface visualization of massive datasets on commodity off-the-shelf clusters

Tomographic imaging and computer simulations are increasingly yielding massive datasets. Interactive and exploratory visualizations have rapidly become indispensable tools to study large volumetric imaging and simulation data. Our scalable isosurface visualization framework on commodity off-the-shelf clusters is an end-to-end parallel and progressive platform, from initial data access to the final display. Interactive browsing of extracted isosurfaces is made possible by using parallel isosurface extraction, and rendering in conjunction with a new specialized piece of image compositing hardware called Metabuffer. In this paper, we focus on the back end scalability by introducing a fully parallel and out-of-core isosurface extraction algorithm. It achieves scalability by using both parallel and out-of-core processing and parallel disks. It statically partitions the volume data to parallel disks with a balanced workload spectrum, and builds I/O-optimal external interval trees to minimize the number of I/O operations of loading large data from disk. We also describe an isosurface compression scheme that is efficient for progress extraction, transmission and storage of isosurfaces.

[1]  Valerio Pascucci,et al.  Fast isocontouring for improved interactivity , 1996, VVS '96.

[2]  Gabriel Taubin,et al.  Geometric compression through topological surgery , 1998, TOGS.

[3]  Sridhar Ramaswamy,et al.  Indexing for Data Models with Constraints and Classes , 1996, J. Comput. Syst. Sci..

[4]  Chandrajit L. Bajaj,et al.  Automatic reconstruction of surfaces and scalar fields from 3D scans , 1995, SIGGRAPH.

[5]  Prabhakar Raghavan,et al.  Probabilistic construction of deterministic algorithms: Approximating packing integer programs , 1986, 27th Annual Symposium on Foundations of Computer Science (sfcs 1986).

[6]  Jeffrey Scott Vitter,et al.  Optimal dynamic interval management in external memory , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[7]  Leslie G. Valiant,et al.  A bridging model for parallel computation , 1990, CACM.

[8]  V. Pascucci,et al.  Parallel accelerated isocontouring for out-of-core visualization , 1999, Proceedings 1999 IEEE Parallel Visualization and Graphics Symposium (Cat. No.99EX381).

[9]  Santiago V. Lombeyda,et al.  Scalable interactive volume rendering using off-the-shelf components , 2001, Proceedings IEEE 2001 Symposium on Parallel and Large-Data Visualization and Graphics (Cat. No.01EX520).

[10]  Pat Hanrahan,et al.  A distributed graphics system for large tiled displays , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[11]  Yossi Matias,et al.  Can shared-memory model serve as a bridging model for parallel computation? , 1997, SPAA '97.

[12]  Charles D. Hansen,et al.  Massively parallel isosurface extraction , 1992, Proceedings Visualization '92.

[13]  Jeffrey Scott Vitter,et al.  External memory algorithms and data structures , 1999, External Memory Algorithms.

[14]  Vijaya Ramachandran,et al.  QSM: A General Purpose Shared-Memory Model for Parallel Computation , 1997, FSTTCS.

[15]  William J. Schroeder,et al.  Interactive out-of-core isosurface extraction , 1998 .

[16]  Michael Deering,et al.  Geometry compression , 1995, SIGGRAPH.

[17]  Gordon Stoll,et al.  Lightning-2: a high-performance display subsystem for PC clusters , 2001, SIGGRAPH.

[18]  Xiaoyu Zhang,et al.  Scalable isosurface visualization of massive datasets on COTS clusters , 2001, Proceedings IEEE 2001 Symposium on Parallel and Large-Data Visualization and Graphics (Cat. No.01EX520).

[19]  William E. Lorensen,et al.  Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[20]  Insung Ihm,et al.  Wavelet‐Based 3D Compression Scheme for Interactive Visualization of Very Large Volume Data , 1999, Comput. Graph. Forum.

[21]  Kenneth Moreland,et al.  Sort-last parallel rendering for viewing extremely large data sets on tile displays , 2001, Proceedings IEEE 2001 Symposium on Parallel and Large-Data Visualization and Graphics (Cat. No.01EX520).

[22]  David Levin,et al.  Progressive Compression of Arbitrary Triangular Meshes , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[23]  Yi-Jen Chiang,et al.  I/O optimal isosurface extraction , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[24]  Henry Fuchs,et al.  A sorting classification of parallel rendering , 1994, IEEE Computer Graphics and Applications.

[25]  Kenneth I. Joy,et al.  Using Difference Intervals for Time-Varying Isosurface Visualization , 2006, IEEE Transactions on Visualization and Computer Graphics.

[26]  Ian H. Witten,et al.  Arithmetic coding for data compression , 1987, CACM.

[27]  Hans-Peter Seidel,et al.  Faster isosurface ray tracing using implicit KD-trees , 2005, IEEE Transactions on Visualization and Computer Graphics.

[28]  Roberto Scopigno,et al.  A modified look-up table for implicit disambiguation of Marching Cubes , 1994, The Visual Computer.

[29]  Homan Igehy,et al.  Pomegranate: a fully scalable graphics architecture , 2000, SIGGRAPH.

[30]  Kwan-Liu Ma,et al.  Parallel Volume Rendering Using Binary-Swap Image Composition , 2008, International Conference on Computer Graphics and Interactive Techniques.

[31]  Charles Hansen,et al.  Parallel Multipipe Rendering for Very Large Isosurface Visualization , 1999 .

[32]  Kwan-Liu Ma,et al.  Parallel volume rendering using binary-swap compositing , 1994, IEEE Computer Graphics and Applications.

[33]  Ulrich Neumann Communication costs for parallel volume-rendering algorithms , 1994, IEEE Computer Graphics and Applications.

[34]  Pat Hanrahan,et al.  Distributed Rendering for Scalable Displays , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[35]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[36]  Valerio Pascucci,et al.  Single Resolution Compression of Arbitrary Triangular Meshes with Properties , 1999, Data Compression Conference.

[37]  Jeffrey Scott Vitter,et al.  External memory algorithms and data structures: dealing with massive data , 2001, CSUR.

[38]  Valerio Pascucci,et al.  The contour spectrum , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[39]  Insung Ihm,et al.  Wavelet-Based 3D Compression Scheme for Very Large Volume Data , 1998, Graphics Interface.

[40]  Thomas A. Funkhouser,et al.  Parallel rendering with K-way replication , 2001, Proceedings IEEE 2001 Symposium on Parallel and Large-Data Visualization and Graphics (Cat. No.01EX520).

[41]  Carl Mueller,et al.  The sort-first rendering architecture for high-performance graphics , 1995, I3D '95.

[42]  Cláudio T. Silva,et al.  External memory techniques for isosurface extraction in scientific visualization , 1998, External Memory Algorithms.

[43]  John S. Montrym,et al.  InfiniteReality: a real-time graphics system , 1997, SIGGRAPH.

[44]  A. Heirich,et al.  Scalable distributed visualization using off-the-shelf components , 1999, Proceedings 1999 IEEE Parallel Visualization and Graphics Symposium (Cat. No.99EX381).

[45]  Charles D. Hansen,et al.  Isosurfacing in span space with utmost efficiency (ISSUE) , 1996, Proceedings of Seventh Annual IEEE Visualization '96.

[46]  B. H. McCormick,et al.  Visualization in scientific computing , 1995 .

[47]  Cláudio T. Silva,et al.  I/O optimal isosurface extraction (extended abstract) , 1997, VIS '97.

[48]  Anselmo Lastra,et al.  PixelFlow: the realization , 1997, HWWS '97.

[49]  Steven Molnar,et al.  Image-Composition Architectures for Real-Time Image Generation , 1991 .

[50]  Thomas W. Crockett,et al.  PARALLEL RENDERING , 1995 .

[51]  Mike M. Chow Optimized geometry compression for real-time rendering , 1997 .

[52]  Valerio Pascucci,et al.  Progressive compression and transmission of arbitrary triangular meshes , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[53]  Sridhar Ramaswamy,et al.  Indexing for data models with constraints and classes (extended abstract) , 1993, PODS '93.

[54]  Peter-Pike J. Sloan,et al.  Interactive ray tracing for isosurface rendering , 1998 .

[55]  Thomas A. Funkhouser,et al.  Load balancing for multi-projector rendering systems , 1999, Workshop on Graphics Hardware.

[56]  John H. Reif,et al.  Fast and compact volume rendering in the compressed transform domain , 1997, Proceedings DCC '97. Data Compression Conference.

[57]  Yoshua Bengio,et al.  The Z-coder adaptive binary coder , 1998, Proceedings DCC '98 Data Compression Conference (Cat. No.98TB100225).

[58]  Edward G. Coffman,et al.  Approximation algorithms for bin packing: a survey , 1996 .

[59]  Thomas A. Funkhouser,et al.  Hybrid sort-first and sort-last parallel rendering with a cluster of PCs , 2000, Workshop on Graphics Hardware.

[60]  Homan Igehy,et al.  The design of a parallel graphics interface , 1998, SIGGRAPH.