Feverfew: a scalable coverage-based hybrid overlay for Internet-scale pub/sub networks

The publish/subscribe (pub/sub) paradigm is a popular communication model for data dissemination in large-scale distributed networks. However, scalability comes with a contradiction between the delivery latency and the memory cost. On one hand, constructing a separate overly per topic guarantees real-time dissemination, while the number of node degrees rapidly increases with the number of subscriptions. On the other hand, maintaining a bounded number of connections per node guarantees small memory cost, while each message has to traverse a large number of uninterested nodes before reaching the subscribers. In this paper, we propose Feverfew, a coverage-based hybrid overlay that disseminates messages to all subscribers without uninterested nodes involved in, and increases the average number of node connections slowly with an increase in the number of subscribers and nodes. The major novelty of Feverfew lies in its heuristic coverage mechanism implemented by combining a gossip-based sampling protocol with a probabilistic searching protocol. Based on the practical workload, our experimental results show that Feverfew significantly outperforms existing coverage-based overlay and DHT-based overlay in various dynamic network environments.

[1]  Sasu Tarkoma,et al.  MOVE: A Large Scale Keyword-Based Content Filtering and Dissemination System , 2012, 2012 IEEE 32nd International Conference on Distributed Computing Systems.

[2]  Mark Handley,et al.  Application-Level Multicast Using Content-Addressable Networks , 2001, Networked Group Communication.

[3]  Roberto Beraldi,et al.  TERA: topic-based event routing for peer-to-peer architectures , 2007, DEBS '07.

[4]  Márk Jelasity,et al.  Gossip-based aggregation in large dynamic networks , 2005, TOCS.

[5]  Andréa W. Richa,et al.  Minimum Maximum Degree Publish-Subscribe Overlay Network Design , 2009, IEEE INFOCOM 2009.

[6]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[7]  Saikat Guha,et al.  Quasar: a probabilistic publish-subscribe system for social networks , 2008, IPTPS.

[8]  Béla Bollobás,et al.  Random Graphs: Notation , 2001 .

[9]  Miguel Castro,et al.  Scribe: a large-scale and decentralized application-level multicast infrastructure , 2002, IEEE J. Sel. Areas Commun..

[10]  Amir H. Payberah,et al.  Vitis: A Gossip-based Hybrid Overlay for Internet-scale Publish/Subscribe Enabling Rendezvous Routing in Unstructured Overlay Networks , 2011, 2011 IEEE International Parallel & Distributed Processing Symposium.

[11]  Anne-Marie Kermarrec,et al.  Rappel: Exploiting interest and network locality to improve fairness in publish-subscribe systems , 2009, Comput. Networks.

[12]  Márk Jelasity,et al.  T-Man: Gossip-based fast overlay topology construction , 2009, Comput. Networks.

[13]  Yoav Tock,et al.  Constructing scalable overlays for pub-sub with many topics , 2007, PODC '07.

[14]  Guruduth Banavar,et al.  Gryphon: An Information Flow Based Approach to Message Brokering , 1998, ArXiv.

[15]  Ming Li,et al.  A Scalable and Elastic Publish/Subscribe Service , 2011, 2011 IEEE International Parallel & Distributed Processing Symposium.

[16]  Yoav Tock,et al.  SpiderCast: a scalable interest-aware overlay for topic-based pub/sub communication , 2007, DEBS '07.

[17]  Béla Bollobás,et al.  Random Graphs , 1985 .

[18]  ZhangZengbin,et al.  Unbiased sampling in directed social graph , 2010 .

[19]  Idit Keidar,et al.  Araneola: a scalable reliable multicast system for dynamic environments , 2004, Third IEEE International Symposium on Network Computing and Applications, 2004. (NCA 2004). Proceedings..

[20]  Xiaoling Li,et al.  A survey of queries over uncertain data , 2013, Knowledge and Information Systems.

[21]  Richard M. Karp,et al.  Randomized rumor spreading , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[22]  David S. Rosenblum,et al.  Achieving scalability and expressiveness in an Internet-scale event notification service , 2000, PODC '00.

[23]  Yijie Wang,et al.  Research and performance evaluation of data replication technology in distributed storage systems , 2006, Comput. Math. Appl..

[24]  Jie Xu,et al.  Internet-based Virtual Computing Environment: Beyond the data center as a computer , 2013, Future Gener. Comput. Syst..

[25]  Maarten van Steen,et al.  CYCLON: Inexpensive Membership Management for Unstructured P2P Overlays , 2005, Journal of Network and Systems Management.