High-performance remote data access for remote visualization

Visualization of remote data needs software that is fundamentally designed for distributed processing and high-speed networks. Motivated by and following the requirements of a distributed visualization application, this article describes the design and implementation of a fast and configurable remote data access system called eavivdata. Because wide-area networks may have a high latency, the remote data access system uses an architecture that effectively hides latency. Four remote data access architectures are analyzed and the results show that an architecture that combines bulk and pipeline processing is the best solution for high-throughput remote data access. The resulting system, also supporting high-speed transport protocols and configurable remote operations, is up to 400 times faster than a comparable existing remote data access system.

[1]  Alessandro Bassi,et al.  The Internet Backplane Protocol: A Study in Resource Sharing , 2002, 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02).

[2]  Leslie G. Butler,et al.  Burning Issues in Tomography Analysis , 2008, Computing in Science & Engineering.

[3]  Douglas Thain,et al.  Parrot: Transparent User-Level Middleware for Data-Intensive Computing , 2005, Scalable Comput. Pract. Exp..

[4]  William E. Allcock,et al.  The Globus Striped GridFTP Framework and Server , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[5]  John Shalf,et al.  The Grid and Future Visualization System Architectures , 2003, IEEE Computer Graphics and Applications.

[6]  Mario Lauria,et al.  Improving the Throughput of Remote Storage Access through Pipelining , 2002, GRID.

[7]  Hans-Christian Hege,et al.  Interactive exploration of large remote micro-CT scans , 2004, IEEE Visualization 2004.

[8]  Joel H. Saltz,et al.  Distributed processing of very large datasets with DataCutter , 2001, Parallel Comput..

[9]  Renato Recio,et al.  A Remote Direct Memory Access Protocol Specification , 2007, RFC.

[10]  M. Livny,et al.  PARROT: AN APPLICATION ENVIRONMENT FOR DATA-INTENSIVE COMPUTING ((PREPRINT VERSION)) , 2005 .

[11]  Robert L. Grossman,et al.  UDT: UDP-based data transfer for high-speed wide area networks , 2007, Comput. Networks.

[12]  Bernard J. Kelly,et al.  Binary Black Holes, Gravitational Waves, and Numerical Relativity , 2007 .

[13]  Jonathan M. Smith Fighting Physics: A Tough Battle , 2009, ACM Queue.

[14]  Sally Floyd,et al.  HighSpeed TCP for Large Congestion Windows , 2003, RFC.