ElastO : Dynamic , Efficient , and Robust Maintenance of Low Fan-out Overlays for Topic-based Publish / Subscribe under Churn

We propose, ElastO, a distributed system for constructing and maintaining scalable churn-resistant overlay networks for topic-based publish/subscribe (pub/sub) systems. ElastO is designed to dynamically tread the balance among several key dimensions: (a) topic-connected overlay (TCO), i.e., the sub-overlay induced by nodes interested in any topic is connected, (b) low maximum and average node fan-outs, (c) high efficiency to maintain the overlay in presence of churn, (d) balanced computation and communication overhead across all the nodes. Existing approaches for maintaining pub/sub TCOs are either static and runtime-costly algorithms, or decentralized protocols that produce significantly higher node degrees. One main challenge is to effectively overcome departure of nodes central to the TCO. ElastO carefully maintains local view at each node and efficiently computes sets of shadow nodes upon churn events, so that all links adjacent to a failed node can be quickly replaced by adding links among the shadow nodes. We evaluate ElastO using both synthetical pub/sub workloads and practical workloads extracted from Facebook and Twitter, and using realworld cluster churn traces released by Google. We show analytically and experimentally that ElastO achieves low fan-out close to static algorithms and high efficiency comparable to decentralized protocols.

[1]  Márk Jelasity,et al.  T-Man: Gossip-based fast overlay topology construction , 2009, Comput. Networks.

[2]  Márk Jelasity,et al.  PeerSim: A scalable P2P simulator , 2009, 2009 IEEE Ninth International Conference on Peer-to-Peer Computing.

[3]  Vasek Chvátal,et al.  A Greedy Heuristic for the Set-Covering Problem , 1979, Math. Oper. Res..

[4]  GhemawatSanjay,et al.  The Google file system , 2003 .

[5]  Maarten van Steen,et al.  PolderCast: Fast, Robust, and Scalable Architecture for P2P Topic-Based Pub/Sub , 2012, Middleware.

[6]  Péter Urbán,et al.  Definition and specification of accrual failure detectors , 2005, 2005 International Conference on Dependable Systems and Networks (DSN'05).

[7]  Prashant Malik,et al.  Cassandra: a decentralized structured storage system , 2010, OPSR.

[8]  Mouna Allani,et al.  Chams: Churn-aware overlay construction for media streaming , 2012, Peer-to-Peer Netw. Appl..

[9]  Yoav Tock,et al.  Hierarchical Clustering of Message Flows in a Multicast Data Dissemination System , 2005, IASTED PDCS.

[10]  Andréa W. Richa,et al.  Parameterized Maximum and Average Degree Approximation in Topic-Based Publish-Subscribe Overlay Network Design , 2009, 2010 IEEE 30th International Conference on Distributed Computing Systems.

[11]  Yoav Tock,et al.  Constructing scalable overlays for pub-sub with many topics , 2007, PODC '07.

[12]  Ben Y. Zhao,et al.  User interactions in social networks and their implications , 2009, EuroSys '09.

[13]  Dorit S. Hochbaum,et al.  Approximation Algorithms for the Set Covering and Vertex Cover Problems , 1982, SIAM J. Comput..

[14]  David R. Karger,et al.  Analysis of the evolution of peer-to-peer systems , 2002, PODC '02.

[15]  Hosung Park,et al.  What is Twitter, a social network or a news media? , 2010, WWW '10.

[16]  João Leitão,et al.  HyParView: A Membership Protocol for Reliable Gossip-Based Broadcast , 2007, 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07).

[17]  Anne-Marie Kermarrec,et al.  The Peer Sampling Service: Experimental Evaluation of Unstructured Gossip-Based Implementations , 2004, Middleware.

[18]  Miguel Castro,et al.  Scribe: a large-scale and decentralized application-level multicast infrastructure , 2002, IEEE J. Sel. Areas Commun..

[19]  Peter Druschel,et al.  Pastry: Scalable, distributed object location and routing for large-scale peer-to- , 2001 .

[20]  Amir H. Payberah,et al.  Vitis: A Gossip-based Hybrid Overlay for Internet-scale Publish/Subscribe Enabling Rendezvous Routing in Unstructured Overlay Networks , 2011, 2011 IEEE International Parallel & Distributed Processing Symposium.

[21]  Emin Gün Sirer,et al.  Client behavior and feed characteristics of RSS, a publish-subscribe system for web micronews , 2005, IMC '05.

[22]  Yoav Tock,et al.  SpiderCast: a scalable interest-aware overlay for topic-based pub/sub communication , 2007, DEBS '07.

[23]  Patrick Th. Eugster,et al.  Data-aware multicast , 2004, International Conference on Dependable Systems and Networks, 2004.