Application-Level Congestion Control Mechanisms for Large Scale Data Transfers Across Computational Grids

In this paper, we discuss the development of a highly efficient, application-level data transfer system for computational grids. We describe and evaluate two application-level congestion control mechanisms, and compare their performance with respect to each other and with that obtained by TCP. We show that both application-level protocols are able to obtain performance that is very close to the maximum available bandwidth while keeping data loss rates low (from 0.06% to 1.6 %). Also, we show that both approaches obtain throughput that is close to an order of magnitude greater than that achieved by TCP. Finally, we begin to address the important issue of whether all data loss should be assumed to represent a network congestion event, with the goal of crafting the response to observed data loss as a function of the root causes of such loss.

[1]  Jason Leigh,et al.  Reliable Blast UDP : predictable high performance bulk data transfer , 2002, Proceedings. IEEE International Conference on Cluster Computing.

[2]  Robert L. Grossman,et al.  Simple Available Bandwidth Utilization Library for High-Speed Wide Area Networks , 2005, The Journal of Supercomputing.

[3]  Robert L. Grossman,et al.  PSockets: The Case for Application-level Network Striping for Data Intensive Applications using High Speed Wide Area Networks , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[4]  Rashid Ansari,et al.  Adaptive Networking for Tele-Immersion , 2001, EGVE/IPT.

[5]  Ian T. Foster,et al.  Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing , 2001, 2001 Eighteenth IEEE Symposium on Mass Storage Systems and Technologies.

[6]  Sally Floyd,et al.  2 What ’ s the Problem ? 2 . 1 Basics TCP uses the following algorithm to adjust its congestion window , 2002 .

[7]  Wu-chun Feng,et al.  The Failure of TCP in High-Performance Computational Grids , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[8]  William Gropp,et al.  High performance wide area data transfers over high performance networks , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[9]  William Gropp,et al.  An evaluation of object-based data transfers on high performance networks , 2002, Proceedings 11th IEEE International Symposium on High Performance Distributed Computing.