OMen: overlay mending for topic-based publish/subscribe systems under churn

We propose, OMen, a distributed system for dynamically maintaining overlays for topic-based publish/subscribe (pub/sub) systems. In particular, OMen supports churn-resistant construction of topic-connected overlays (TCO), which organizes all nodes interested in the same topic in a directly connected dissemination sub-overlay. While aiming at pub/sub deployments in data centers, OMen internally leverages selected peer-to-peer technologies, such as T-Man as the underlying topology maintenance protocol. Existing approaches for constructing pub/sub TCOs are (i) centralized algorithms that guarantee low node degrees at the cost of prohibitive running time and (ii) decentralized protocols that are time efficient while lacking bounds on node degrees. We show both analytically and experimentally that OMen combines the best from both worlds. Namely, OMen achieves (i) low node degrees, close to centralized algorithms, and (ii) high efficiency, scalability, and load balance, comparable to decentralized protocols. Our evaluation uses both synthetic pub/sub workloads and real-world ones extracted from Facebook and Twitter. We generate churn traces with Google cluster data.

[1]  Hans-Arno Jacobsen,et al.  Scaling Construction of Low Fan-out Overlays for Topic-Based Publish/Subscribe Systems , 2011, 2011 31st International Conference on Distributed Computing Systems.

[2]  John Kubiatowicz,et al.  Handling churn in a DHT , 2004 .

[3]  Rizal Setya Perdana.  What is Twitter , 2013 .

[4]  Yoav Tock,et al.  Hierarchical Clustering of Message Flows in a Multicast Data Dissemination System , 2005, IASTED PDCS.

[5]  Andréa W. Richa,et al.  Parameterized maximum and average degree approximation in topic-based publish-subscribe overlay network design , 2016, Comput. Networks.

[6]  Ben Y. Zhao,et al.  User interactions in social networks and their implications , 2009, EuroSys '09.

[7]  Yoav Tock,et al.  Design of Routing Protocols and Overlay Topologies for Topic-based Publish/Subscribe on Small-World Networks , 2015, Middleware Industry.

[8]  Anne-Marie Kermarrec,et al.  Rappel: Exploiting interest and network locality to improve fairness in publish-subscribe systems , 2009, Comput. Networks.

[9]  Jon M. Kleinberg,et al.  The small-world phenomenon: an algorithmic perspective , 2000, STOC '00.

[10]  John K. Ousterhout,et al.  In Search of an Understandable Consensus Algorithm , 2014, USENIX ATC.

[11]  Andréa W. Richa,et al.  Parameterized Maximum and Average Degree Approximation in Topic-Based Publish-Subscribe Overlay Network Design , 2009, 2010 IEEE 30th International Conference on Distributed Computing Systems.

[12]  João Leitão,et al.  HyParView: A Membership Protocol for Reliable Gossip-Based Broadcast , 2007, 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07).

[13]  Pascal Felber,et al.  Scaling Up Publish/Subscribe Overlays Using Interest Correlation for Link Sharing , 2013, IEEE Transactions on Parallel and Distributed Systems.

[14]  Amir H. Payberah,et al.  Vitis: A Gossip-based Hybrid Overlay for Internet-scale Publish/Subscribe Enabling Rendezvous Routing in Unstructured Overlay Networks , 2011, 2011 IEEE International Parallel & Distributed Processing Symposium.

[15]  Anne-Marie Kermarrec,et al.  The Peer Sampling Service: Experimental Evaluation of Unstructured Gossip-Based Implementations , 2004, Middleware.

[16]  Márk Jelasity,et al.  T-Man: Gossip-based fast overlay topology construction , 2009, Comput. Networks.

[17]  Maarten van Steen,et al.  PolderCast: Fast, Robust, and Scalable Architecture for P2P Topic-Based Pub/Sub , 2012, Middleware.

[18]  Yoav Tock,et al.  Constructing scalable overlays for pub-sub with many topics , 2007, PODC '07.

[19]  Yoav Tock,et al.  Overlay Design for Topic-Based Publish/Subscribe under Node Degree Constraints , 2016, 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS).

[20]  Hosung Park,et al.  What is Twitter, a social network or a news media? , 2010, WWW '10.

[21]  Miguel Castro,et al.  Scribe: a large-scale and decentralized application-level multicast infrastructure , 2002, IEEE J. Sel. Areas Commun..

[22]  Yoav Tock,et al.  SpiderCast: a scalable interest-aware overlay for topic-based pub/sub communication , 2007, DEBS '07.

[23]  Hans-Arno Jacobsen,et al.  PNUTS: Yahoo!'s hosted data serving platform , 2008, Proc. VLDB Endow..

[24]  Vasek Chvátal,et al.  A Greedy Heuristic for the Set-Covering Problem , 1979, Math. Oper. Res..

[25]  Márk Jelasity,et al.  PeerSim: A scalable P2P simulator , 2009, 2009 IEEE Ninth International Conference on Peer-to-Peer Computing.

[26]  Leslie Lamport,et al.  The part-time parliament , 1998, TOCS.

[27]  Roberto Beraldi,et al.  TERA: topic-based event routing for peer-to-peer architectures , 2007, DEBS '07.

[28]  Seif Haridi,et al.  Efficient Broadcast in Structured P2P Networks , 2003, IPTPS.

[29]  Emin Gün Sirer,et al.  Client behavior and feed characteristics of RSS, a publish-subscribe system for web micronews , 2005, IMC '05.

[30]  Yoav Tock,et al.  Magnet: practical subscription clustering for Internet-scale publish/subscribe , 2010, DEBS '10.

[31]  Dorit S. Hochbaum,et al.  Approximation Algorithms for the Set Covering and Vertex Cover Problems , 1982, SIAM J. Comput..

[32]  Prashant Malik,et al.  Cassandra: a decentralized structured storage system , 2010, OPSR.