MOB: zero-configuration high-throughput multicasting for grid applications

Grid applications often need to distribute large amounts of data efficiently from one cluster to multiple others (multicast). Existing methods usually arrange nodes in optimized tree structures, based on external network monitoring data. This dependence on monitoring data, however, severely impacts both ease of deployment and adaptivity to dynamically changing network conditions. In this paper, we present Multicast Optimizing Bandwidth (MOB), a high-throughput multicast approach, inspired by the BitTorrent protocol. With MOB, data transfers are initiated by the receivers that try to steal data from peer clusters. Instead of using potentially outdated monitoring data, MOB automatically adapts to the currently achievable bandwidth ratios. Our experimental evaluation compares MOB to both the BitTorrent protocol and to our previous approach, Balanced Multicasting, the latter optimizing multicast trees based on external monitoring data. We show that MOB outperforms the BitTorrent protocol. MOB is competitive with Balanced Multicasting as long as the network bandwidth remains stable. With dynamically changing bandwith, MOB outperforms Balanced Multicasting by wide margins.

[1]  S. Hemminger.  Network Emulation with NetEm , 2022 .

[2]  Antonio J. Plaza,et al.  Commodity cluster-based parallel processing of hyperspectral imagery , 2006, J. Parallel Distributed Comput..

[3]  B. Cohen,et al.  Incentives Build Robustness in Bit-Torrent , 2003 .

[4]  Rauf Izmailov,et al.  Fast Parallel File Replication in Data Grid , 2004 .

[5]  Jason Maassen,et al.  Ibis: a flexible and efficient Java-based Grid programming environment: Research Articles , 2005 .

[6]  Jason Maassen,et al.  Smartsockets: solving the connectivity problems in grid computing , 2007, HPDC '07.

[7]  Richard G. Baraniuk,et al.  pathChirp: Efficient available bandwidth estimation for network paths , 2003 .

[8]  Klara Nahrstedt,et al.  Maxmin overlay multicast: rate allocation and tree construction , 2004, Twelfth IEEE International Workshop on Quality of Service, 2004. IWQOS 2004..

[9]  Reuven Cohen,et al.  A unicast-based approach for streaming multicast , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[10]  Jason Maassen,et al.  Ibis: a flexible and efficient Java‐based Grid programming environment , 2005, Concurr. Pract. Exp..

[11]  Bronis R. de Supinski,et al.  Exploiting hierarchy in parallel computer networks to optimize collective operation performance , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[12]  Richard Wolski,et al.  Experiences with predicting resource performance on-line in computational grid settings , 2003, PERV.

[13]  Parameswaran Ramanathan,et al.  What do packet dispersion techniques measure? , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[14]  Alexandru Iosup,et al.  Tribler: A social-based based peer to peer system , 2006 .

[15]  Jason Maassen,et al.  Middleware adaptation with the Delphoi service , 2006, Concurr. Comput. Pract. Exp..

[16]  Marcel Worring,et al.  High-Performance Distributed Image and Video Content Analysis with Parallel-Horus , 2007 .

[17]  Henri E. Bal,et al.  MagPIe: MPI's collective communication operations for clustered wide area systems , 1999, PPoPP '99.

[18]  GWD-C.  A Hierarchy of Network Performance Characteristics for Grid Applications and Services , 2003 .

[19]  Min Sik Kim,et al.  Optimal distribution tree for Internet streaming media , 2003, 23rd International Conference on Distributed Computing Systems, 2003. Proceedings..

[20]  Miguel Castro,et al.  SplitStream: high-bandwidth multicast in cooperative environments , 2003, SOSP '03.

[21]  Yves Robert,et al.  Broadcast trees for heterogeneous platforms , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[22]  Kees Verstoep,et al.  Network performance-aware collective communication for clustered wide-area systems , 2001, Parallel Comput..

[23]  Klara Nahrstedt,et al.  On achieving optimized capacity utilization in application overlay networks with multiple competing sessions , 2004, SPAA '04.

[24]  Kees Verstoep,et al.  Efficient reliable multicast on Myrinet , 1996, Proceedings of the 1996 ICPP Workshop on Challenges for Parallel Processing.

[25]  William Chan,et al.  Improving Traffic Locality in BitTorrent via Biased Neighbor Selection , 2006, 26th IEEE International Conference on Distributed Computing Systems (ICDCS'06).

[26]  Thomas R. Gross,et al.  Design, Implementation, and Evaluation of the Remos Network Monitoring System , 2004, Journal of Grid Computing.