论文信息 - Scalable Distributed Execution Environment for Large Data Visualization

Scalable Distributed Execution Environment for Large Data Visualization

To use heterogeneous and geographically distributed resources as a platform for parallel visualization is an intriguing topic of research. This is because of the immense potential impact of the work, and also because of its use of a full range of challenging technologies. In this work, we designed an execution environment for visualization of massive scientific datasets, using network functional units (NFU) for processing power, logistical networking for storage management and visualization cookbook library (vcblib) for visualization operations. This environment is based solely on computers distributed across the Internet that are owned and operated by independent institutions, while being openly shared for free. Those Internet computers are inherently of heterogeneous hardware configuration and running a variety of operating systems. Using 100 such processors, we have been able to obtain the same level of performance offered by a 64-node cluster of 2.2 GHz P4 processors, while processing a 75GBs subset of a cutting-edge simulation dataset. Due to its inherently shared nature, this execution environment for data-intensive visualization could provide a viable means of collaboration among geographically separated users.

Jian Huang | Micah Beck | Terry Moore | Huadong Liu

[1] Amy Henderson Squilacote. The Paraview Guide , 2008 .

[2] Klaus Mueller,et al. A practical evaluation of popular volume rendering algorithms , 2000, VVS '00.

[3] Thomas A. Funkhouser,et al. Parallel rendering with K-way replication , 2001, Proceedings IEEE 2001 Symposium on Parallel and Large-Data Visualization and Graphics (Cat. No.01EX520).

[4] William E. Lorensen,et al. Marching cubes: A high resolution 3D surface construction algorithm , 1987, SIGGRAPH.

[5] Micah Beck,et al. An end-to-end approach to globally scalable network storage , 2002, SIGCOMM '02.

[6] Jerome H. Saltzer,et al. End-to-end arguments in system design , 1984, TOCS.

[7] Jian Huang,et al. Dynamic co-scheduling of distributed computation and replication , 2006, Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06).

[8] Micah Beck,et al. An end-to-end approach to globally scalable programmable networking , 2003, FDNA '03.

[9] Amy Henderson,et al. The ParaView Guide: A Parallel Visualization Application , 2004 .

[10] John Shalf,et al. Cactus and Visapult: An ultra-high performance grid-distributed visualization architecture using connectionless protocols , 2002 .

[11] John Shalf,et al. Ieee Computer Graphics and Applications Numerical Relativity Grid-distributed Visualizations Using Connectionless Protocols Graphics Applications for Grid Computing , 2022 .