In Search of Simplicity: A Self-Organizing Multi-Source Multicast Overlay

Multicast communication primitives have broad utility as building blocks for distributed applications. The challenge is to create and maintain the distributed structures that support these primitives while accounting for volatile end nodes and variable network characteristics. Most solutions proposed to date rely on complex algorithms or global information, thus limiting the scale of deployments and acceptance outside the academic realm. This article introduces a low-complexity, self organizing solution for maintaining multicast trees, that we refer to as UMM (Unstructured Multi-source Multicast). UMM uses traditional distributed systems techniques: layering, soft-state, and passive data collection to adapt to the dynamics of the physical network and maintain data dissemination trees. The result is a simple, adaptive system with lower overheads than more complex alternatives. We have implemented UMM and evaluated it on a 100-node PlanetLab testbed and on up to 1024-node emulated ModelNet networks Extensive experimental evaluations demonstrate UMM's low overhead, efficient network usage compared to alternative solutions, and ability to quickly adapt to network changes and to recover from failures.

[1]  FaloutsosMichalis,et al.  On power-law relationships of the Internet topology , 1999 .

[2]  Ratul Mahajan,et al.  A Study of the Performance Potential of DHT-based Overlays , 2003, USENIX Symposium on Internet Technologies and Systems.

[3]  Mark Handley,et al.  Topologically-aware overlay construction and server selection , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[4]  Stefan Savage,et al.  Understanding Availability , 2003, IPTPS.

[5]  Fabián E. Bustamante,et al.  Friendships that Last: Peer Lifespan and its Role in P2P Protocols , 2003, WCW.

[6]  Amin Vahdat,et al.  Bullet: high bandwidth data dissemination using an overlay mesh , 2003, SOSP '03.

[7]  Robert Morris,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM 2001.

[8]  Peter Z. Kunszt,et al.  Giggle: A Framework for Constructing Scalable Replica Location Services , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[9]  Seif Haridi,et al.  Efficient Broadcast in Structured P2P Networks , 2003, IPTPS.

[10]  Mark Handley,et al.  Application-Level Multicast Using Content-Addressable Networks , 2001, Networked Group Communication.

[11]  David E. Culler,et al.  Operating Systems Support for Planetary-Scale Network Services , 2004, NSDI.

[12]  Lixia Zhang,et al.  Host multicast: a framework for delivering multicast to end users , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[13]  Michalis Faloutsos,et al.  On power-law relationships of the Internet topology , 1999, SIGCOMM '99.

[14]  Robert Tappan Morris,et al.  Designing a DHT for Low Latency and High Throughput , 2004, NSDI.

[15]  Ben Y. Zhao,et al.  Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination , 2001, NOSSDAV '01.

[16]  Dinesh C. Verma,et al.  ALMI: An Application Level Multicast Infrastructure , 2001, USITS.

[17]  Pierluigi Crescenzi,et al.  A compendium of NP optimization problems , 1994, WWW Spring 1994.

[18]  Amin Vahdat,et al.  Distributed Resource Discovery on PlanetLab with SWORD , 2004, WORLDS.

[19]  Walid Dabbous,et al.  Multipoint Communication: A Survey of Protocols, Functions, and Mechanisms , 1997, IEEE J. Sel. Areas Commun..

[20]  Aravind Srinivasan,et al.  Resilient multicast using overlays , 2003, IEEE/ACM Transactions on Networking.

[21]  Michal Feldman,et al.  Workshop Report for 2nd International Workshop on Peer-to-Peer Systems (IPTPS '03) 21-22 February 2003 - Claremont Hotel, Berkeley, CA, USA , 2003, IPTPS.

[22]  Ibrahim Matta,et al.  BRITE: an approach to universal topology generation , 2001, MASCOTS 2001, Proceedings Ninth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[23]  Helen J. Wang,et al.  An evaluation of scalable application-level multicast built using peer-to-peer overlays , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[24]  John Kubiatowicz,et al.  Handling churn in a DHT , 2004 .

[25]  David D. Clark,et al.  Architectural considerations for a new generation of protocols , 1990, SIGCOMM '90.

[26]  Azer Bestavros,et al.  Small-world Characteristics of Internet Topologies and Multicast Scaling , 2003 .

[27]  David E. Culler,et al.  The ganglia distributed monitoring system: design, implementation, and experience , 2004, Parallel Comput..

[28]  Simon S. Lam,et al.  Failure recovery for structured P2P networks: protocol design and performance evaluation , 2004, SIGMETRICS '04/Performance '04.

[29]  Amin Vahdat,et al.  MACEDON: Methodology for Automatically Creating, Evaluating, and Designing Overlay Networks , 2004, NSDI.

[30]  Stefan Saroiu,et al.  A Measurement Study of Peer-to-Peer File Sharing Systems , 2001 .

[31]  Stephen E. Deering,et al.  Multicast routing in internetworks and extended LANs , 1988, SIGCOMM '88.

[32]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[33]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM 2001.

[34]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[35]  Deborah Estrin,et al.  Protocol independent multicast-dense mode (pim-dm): protocol specification , 1996 .

[36]  Ian T. Foster,et al.  Mapping the Gnutella Network: Properties of Large-Scale Peer-to-Peer Systems and Implications for System Design , 2002, ArXiv.

[37]  Kai Shen,et al.  Structure Management for Scalable Overlay Service Construction , 2004, NSDI.

[38]  Jerome H. Saltzer,et al.  End-to-end arguments in system design , 1984, TOCS.

[39]  Dejan Kostic,et al.  Scalability and accuracy in a large-scale network emulator , 2002, CCRV.

[40]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[41]  Srinivasan Seshan,et al.  The Impact of Heterogeneous Bandwidth Constraints on DHT-Based Multicast Protocols , 2005, IPTPS.

[42]  Kevin C. Almeroth,et al.  Collecting and modeling the join/leave behavior of multicast group members in the MBone , 1996, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing.

[43]  David Clark,et al.  Architectural considerations for a new generation of protocols , 1990, SIGCOMM 1990.

[44]  Rick Stevens,et al.  Access grid: Immersive group-to-group collaborative visualization , 2000 .

[45]  Christophe Diot,et al.  Deployment issues for the IP multicast service and architecture , 2000, IEEE Netw..

[46]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[47]  Kirk L. Johnson,et al.  Overcast: reliable multicasting with on overlay network , 2000, OSDI.

[48]  Andrew Adams,et al.  Protocol Independent Multicast - Dense Mode (PIM-DM): Protocol Specification (Revised) , 2005, RFC.

[49]  Bobby Bhattacharjee,et al.  Scalable application layer multicast , 2002, SIGCOMM '02.

[50]  Mark Handley,et al.  RFC 5348: TCP Friendly Rate Control (TFRC): Protocol Specification , 2008 .

[51]  Miguel Castro,et al.  SplitStream: high-bandwidth multicast in cooperative environments , 2003, SOSP '03.

[52]  Anthony Young,et al.  Overlay mesh construction using interleaved spanning trees , 2004, IEEE INFOCOM 2004.

[53]  Krishna P. Gummadi,et al.  The impact of DHT routing geometry on resilience and proximity , 2003, SIGCOMM '03.

[54]  Yatin Chawathe Scattercast: an adaptable broadcast distribution framework , 2003, Multimedia Systems.

[55]  Miguel Castro,et al.  Scribe: a large-scale and decentralized application-level multicast infrastructure , 2002, IEEE J. Sel. Areas Commun..

[56]  Jacky C. Chu,et al.  Availability and locality measurements of peer-to-peer file systems , 2002, SPIE ITCom.

[57]  David Moore,et al.  Replication Strategies for Highly Available Peer-to-Peer Storage , 2002, Future Directions in Distributed Computing.

[58]  Joseph D. Touch Overlay Networks , 2004, The Practical Handbook of Internet Computing.

[59]  Srinivasan Seshan,et al.  A case for end system multicast , 2002, IEEE J. Sel. Areas Commun..

[60]  Ian Foster,et al.  A Decentralized, Adaptive, Replica Location Service , 2002 .