Efficient epidemic-style protocols for reliable and scalable multicast

Epidemic-style (gossip-based) techniques have recently emerged as a scalable class of protocols for peer-to-peer reliable multicast dissemination in large process groups. These protocols provide probabilistic guarantees on reliability and scalability. However, popular implementations of epidemic-style dissemination are reputed to suffer from two major drawbacks: (a) (Network Overhead) when deployed on a WAN-wide or VPN-wide scale they generate a large number of packets that transit across the boundaries of multiple network domains (e.g., LANs, subnets, ASs), causing an overload on core network elements such as bridges, routers, and associated links; (b) (Lack of Adaptivity) they impose the same load on process group members and the network even under reduced failure rates (viz., packet losses, process failures). lit this paper we report on the (first) comprehensive set of solutions to these problems. The solution is comprised of two protocols: (1) a hierarchical gossiping protocol, and (2) an adaptive multicast dissemination framework that allows use of any gossiping primitive within it. These protocols work within a virtual peer-to-peer hierarchy called the Leaf Box hierarchy. Processes can be allocated in a topologically aware manner to the leaf boxes of this structure, so that (1) and (2) produce low traffic across domain boundaries in the network. In the interests of space, this paper focuses on a detailed discussion and evaluation (through simulations) of only the hierarchical gossiping protocol. We present an overview of the adaptive dissemination protocol and its properties.

[1]  Kenneth P. Birman,et al.  Bimodal multicast , 1999, TOCS.

[2]  Indranil Gupta,et al.  Fighting fire with fire: using randomized gossip to combat stochastic scalability limits , 2002 .

[3]  Ian T. Foster,et al.  Locating Data in (Small-World?) Peer-to-Peer Scientific Collaborations , 2002, IPTPS.

[4]  Steven McCanne,et al.  A reliable multicast framework for light-weight sessions and application level framing , 1995, SIGCOMM '95.

[5]  Keith Marzullo,et al.  Directional Gossip: Gossip in a Wide Area Network , 1999, EDCC.

[6]  Abhinandan Das,et al.  SWIM: scalable weakly-consistent infection-style process group membership protocol , 2002, Proceedings International Conference on Dependable Systems and Networks.

[7]  Matthew Thomas Lucas,et al.  Efficient data distribution in large-scale multicast networks , 1998 .

[8]  Indranil Gupta,et al.  Scalable fault-tolerant aggregation in large process groups , 2001, 2001 International Conference on Dependable Systems and Networks.

[9]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[10]  Anne-Marie Kermarrec,et al.  Reliable probabilistic communication in large-scale information dissemination systems , 2000 .

[11]  Mark Handley,et al.  Topologically-aware overlay construction and server selection , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[12]  Anne-Marie Kermarrec,et al.  SCAMP: Peer-to-Peer Lightweight Membership Service for Large-Scale Group Communication , 2001, Networked Group Communication.

[13]  Scott Shenker,et al.  Epidemic algorithms for replicated database maintenance , 1988, OPSR.

[14]  Alexander Grey,et al.  The Mathematical Theory of Infectious Diseases and Its Applications , 1977 .

[15]  Patrick Th. Eugster,et al.  Probabilistic multicast , 2002, Proceedings International Conference on Dependable Systems and Networks.

[16]  Keith Marzullo,et al.  Gossip versus Deterministically Constrained Flooding on Small Networks , 2000, DISC.

[17]  Anne-Marie Kermarrec,et al.  Lightweight probabilistic broadcast , 2003, TOCS.

[18]  Robbert van Renesse,et al.  A Gossip-Style Failure Detection Service , 2009 .

[19]  Ellen W. Zegura,et al.  How to model an internetwork , 1996, Proceedings of IEEE INFOCOM '96. Conference on Computer Communications.

[20]  Kenneth P. Birman,et al.  A randomized error recovery algorithm for reliable multicast , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[21]  N. Ling.  The Mathematical Theory of Infectious Diseases and its applications , 1978 .

[22]  Ian Foster,et al.  Locating data in (small-world?) p2p scientific collaborations , 2002 .

[23]  Robbert van Renesse,et al.  Scalable Management and Data Mining Using Astrolabe , 2002, IPTPS.

[24]  ZHANGLi-xia,et al.  A reliable multicast framework for light-weight sessions and application level framing , 1995 .