Progressive Retrieval and Hierarchical Visualization of Large Remote Data

The size of data sets produced on remote supercomputer facilities frequently exceeds the processing capabilities of local visualization workstations. This phenomenon increasingly limits scientists when analyzing results of large-scale scientific simulations. That problem gets even more prominent in scientific collaborations, spanning large virtual organizations, working on common shared sets of data distributed in Grid environments. In the visualization community, this problem is addressed by distributing the visualization pipeline. In particular, early stages of the pipeline are executed on resources closer to the initial (remote) locations of the data sets. This paper presents an efficient technique for placing the first two stages of the visualization pipeline (data access and data filter) onto remote resources. This is realized by exploiting the extended retrieve feature of GridFTP for flexible, high performance access to very large HDF5 files. We reduce the number of network transactions for filtering operations by utilizing a server side data processing plugin, and hence reduce latency overhead compared to GridFTP partial file access. The paper further describes the application of hierarchical rendering techniques on remote uniform data sets, which make use of the remote data

[1]  Joel H. Saltz,et al.  DataCutter: Middleware for Filtering Very Large Scientific Datasets on Archival Storage Systems , 2000, IEEE Symposium on Mass Storage Systems.

[2]  Brian Cabral,et al.  Accelerated volume rendering and tomographic reconstruction using texture mapping hardware , 1994, VVS '94.

[3]  Thomas Ertl,et al.  Level-of-Detail Volume Rendering via 3D Textures , 2000, 2000 IEEE Symposium on Volume Visualization (VV 2000).

[4]  Naren Ramakrishnan,et al.  Data Mining Applications in Bioinformatics , 2001 .

[5]  J. T. Gray,et al.  Hierarchical Large-scale Volume Representation with \(\root 3 \of 2 \) Subdivision and Trivariate B-spline Wavelets , 2004 .

[6]  Ian T. Foster,et al.  A security architecture for computational grids , 1998, CCS '98.

[7]  L.A. Freitag,et al.  Adaptive, Multiresolution Visualization of Large Data Sets using a Distributed Memory Octree , 1999, ACM/IEEE SC 1999 Conference (SC'99).

[8]  John Shalf,et al.  Numerical Relativity in a Distributed Environment , 1999, PPSC.

[9]  G. Allen,et al.  The Cactus Code: a problem solving environment for the grid , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.

[10]  Bernd Hamann,et al.  Interactive Visualization Of Very Large Datasets Using An Out-of-Core Point-based Approach , 2003 .

[11]  Alyn P. Rockwood,et al.  Enabling View-Dependent Progressive Volume Visualization on the Grid , 2003, IEEE Computer Graphics and Applications.

[12]  Werner Benger,et al.  Efficient Distributed File I/O for Visualization in Grid Environments , 2000 .

[13]  Florin Isaila,et al.  Mapping functions and data redistribution for parallel files , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[14]  Steven Tuecke,et al.  GridFTP: Protocol Extensions to FTP for the Grid , 2001 .

[15]  Hans-Christian Hege,et al.  GriKSL - Immersive Überwachung und Steuerung von Simulationen auf entfernten Supercomputern , 2002 .

[16]  Richard Wolski,et al.  The network weather service: a distributed resource performance forecasting service for metacomputing , 1999, Future Gener. Comput. Syst..

[17]  Helmut Pralle,et al.  Lösungsansätze zur Visualisierung im High Performance Computing und Networking Kontext , 2002, DFN-Arbeitstagung über Kommunikationsnetze.

[18]  William J. Schroeder,et al.  A Parallel Approach for Efficiently Visualizing Extremely Large, Time-Varying Datasets , 2000 .

[19]  John Rumble,et al.  Publication and use of large data sets , 2001 .