Network-Aware Epidemic Broadcast

Epidemic multicast is an emerging resilient and scalable approach to the reliable dissemination of application data in the context of very large scale distributed systems. Unfortunately, the resilience and scalability come at the cost of considerable redundancy which led to high resource consumption on both links and nodes. In environments with resource constrained links, such as in Cloud Computing infrastructure composed by data centers organized in a federation around the globe, the high resource consumption precludes the use of this class of protocols. The goal of this dissertation is therefore to cope with the constraints of these scenarios, by reducing the network load imposed on the constrained long distance links. This is achieved by constructing an overlay that re ects the characteristics of the links, and by using a dissemination protocol that takes into account locality when transmitting the message payloads. We conducted an extensive experimental evaluation that presents an improvement over an order of magnitude in the number of messages that traverse the costlier links, without endangering the resilience and scalability properties that make epidemic based protocols so attractive.

[1]  Leslie Lamport,et al.  Reaching Agreement in the Presence of Faults , 1980, JACM.

[2]  Keith Marzullo,et al.  Directional Gossip: Gossip in a Wide Area Network , 1999, EDCC.

[3]  Robbert van Renesse,et al.  Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining , 2003, TOCS.

[4]  Miguel Castro,et al.  Scribe: a large-scale and decentralized application-level multicast infrastructure , 2002, IEEE J. Sel. Areas Commun..

[5]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[6]  Kirk L. Johnson,et al.  Overcast: reliable multicasting with on overlay network , 2000, OSDI.

[7]  Anne-Marie Kermarrec,et al.  From Epidemics to Distributed Computing , 2004 .

[8]  José Pereira,et al.  Rewriting “ The Turtle and the Hare ” : Sleeping to Get There Faster , .

[9]  Anne-Marie Kermarrec,et al.  Probabilistic Reliable Dissemination in Large-Scale Systems , 2003, IEEE Trans. Parallel Distributed Syst..

[10]  Anne-Marie Kermarrec,et al.  HiScamp: self-organizing hierarchical membership protocol , 2002, EW 10.

[11]  Anne-Marie Kermarrec,et al.  Network awareness and failure resilience in self-organizing overlay networks , 2003, 22nd International Symposium on Reliable Distributed Systems, 2003. Proceedings..

[12]  Rui Oliveira,et al.  Self tuning with self confidence , 2008, DSN 2008.

[13]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[14]  Dan Rubenstein,et al.  A lightweight, robust P2P system to handle flash crowds , 2002, IEEE Journal on Selected Areas in Communications.

[15]  N. Ling The Mathematical Theory of Infectious Diseases and its applications , 1978 .

[16]  Mark Handley,et al.  Application-Level Multicast Using Content-Addressable Networks , 2001, Networked Group Communication.

[17]  Christos Gkantsidis,et al.  Random walks in peer-to-peer networks: Algorithms and evaluation , 2006, Perform. Evaluation.

[18]  João Leitão,et al.  HyParView: A Membership Protocol for Reliable Gossip-Based Broadcast , 2007, 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07).

[19]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[20]  Helen J. Wang,et al.  An evaluation of scalable application-level multicast built using peer-to-peer overlays , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[21]  Anne-Marie Kermarrec,et al.  SCAMP: Peer-to-Peer Lightweight Membership Service for Large-Scale Group Communication , 2001, Networked Group Communication.

[22]  Luís E. T. Rodrigues,et al.  Semantically Reliable Multicast: Definition, Implementation, and Performance Evaluation , 2003, IEEE Trans. Computers.

[23]  Ben Y. Zhao,et al.  Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination , 2001, NOSSDAV '01.

[24]  C. Pipper,et al.  [''R"--project for statistical computing]. , 2008, Ugeskrift for laeger.

[25]  Amin Vahdat,et al.  A scalable, commodity data center network architecture , 2008, SIGCOMM '08.

[26]  Richard M. Karp,et al.  Randomized rumor spreading , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[27]  Anne-Marie Kermarrec,et al.  NEEM: network-friendly epidemic multicast , 2003, 22nd International Symposium on Reliable Distributed Systems, 2003. Proceedings..

[28]  Alexander Grey,et al.  The Mathematical Theory of Infectious Diseases and Its Applications , 1977 .

[29]  Yulin Fang,et al.  The pendulum swings back: individual acceptance of re-centralized application platforms , 2006, DATB.

[30]  Boris Koldehofe,et al.  Buffer management in probabilistic peer-to-peer communication protocols , 2003, 22nd International Symposium on Reliable Distributed Systems, 2003. Proceedings..

[31]  Anne-Marie Kermarrec,et al.  Adaptive gossip-based broadcast , 2003, 2003 International Conference on Dependable Systems and Networks, 2003. Proceedings..

[32]  Tatsuhiro Tsuchiya,et al.  Constructing Overlay Networks with Low Link Costs and Short Paths , 2007, Sixth IEEE International Symposium on Network Computing and Applications (NCA 2007).

[33]  Kenneth P. Birman,et al.  Bimodal multicast , 1999, TOCS.

[34]  Anne-Marie Kermarrec,et al.  The Peer Sampling Service: Experimental Evaluation of Unstructured Gossip-Based Implementations , 2004, Middleware.

[35]  Maarten van Steen,et al.  CYCLON: Inexpensive Membership Management for Unstructured P2P Overlays , 2005, Journal of Network and Systems Management.

[36]  Rachid Guerraoui,et al.  Hierarchical Probabilistic Multicast , 2001 .

[37]  Luís E. T. Rodrigues,et al.  Emergent Structure in Unstructured Epidemic Multicast , 2007, 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07).

[38]  Anne-Marie Kermarrec,et al.  Lightweight probabilistic broadcast , 2003, TOCS.