Wide Area Network-scale Discovery and Data Dissemination in Data-centric Publish/Subscribe Systems

Distributed systems found in application domains, such as smart transportation and smart grids, inherently require dissemination of large amount of data over wide area networks (WAN). A large portion of this data is analyzed and used to manage the overall health and safety of these distributed systems. The data-centric, publish/subscribe (pub/sub) paradigm is an attractive choice to address these needs because it provides scalable and loosely coupled data communications. However, existing data-centric pub/sub mechanisms supporting quality of service (QoS) tend to operate effectively only within local area networks. Likewise broker-based solutions that operate at WAN-scale seldom provide mechanisms to coordinate among themselves for discovery and dissemination of information, and cannot handle both the heterogeneity of pub/sub endpoints as well as the significant churn in endpoints that is common in WAN-scale systems. To address these limitations, this paper presents PubSubCoord, which is a cloud-based coordination and discovery service for WAN-scale pub/sub systems. PubSubCoord, which builds upon the ZooKeeper coordination primitives, realizes a WAN-scale, adaptive, and low-latency endpoint discovery and data dissemination architecture by (a) balancing the load using elastic cloud resources, (b) clustering brokers by topics for affinity, and (c) minimizing the number of data delivery hops in the pub/sub overlay.

[1]  Luís E. T. Rodrigues,et al.  Scalable QoS-Based Event Routing in Publish-Subscribe Systems , 2005, Fourth IEEE International Symposium on Network Computing and Applications.

[2]  Juan M. López-Soler,et al.  A content-aware bridging service for publish/subscribe environments , 2013, J. Syst. Softw..

[3]  Yoav Tock,et al.  SpiderCast: a scalable interest-aware overlay for topic-based pub/sub communication , 2007, DEBS '07.

[4]  Steve Vinoski,et al.  Advanced Message Queuing Protocol , 2006, IEEE Internet Computing.

[5]  Luigi Rizzo,et al.  Dummynet: a simple approach to the evaluation of network protocols , 1997, CCRV.

[6]  Jaswinder Pal Singh,et al.  Efficient event routing in content-based publish-subscribe service networks , 2004, IEEE INFOCOM 2004.

[7]  Miguel Castro,et al.  SCRIBE: The Design of a Large-Scale Event Notification Infrastructure , 2001, Networked Group Communication.

[8]  Kurt Rothermel,et al.  PLEROMA: a SDN-based high performance publish/subscribe middleware , 2014, Middleware.

[9]  Douglas C. Schmidt,et al.  Supporting End-to-end Scalability and Real-time Event Dissemination in the OMG Data Distribution Service over Wide Area Networks , 2013 .

[10]  Gordon S. Blair,et al.  GREEN: A Configurable and Re-configurable Publish-Subscribe Middleware for Pervasive Computing , 2005, OTM Conferences.

[11]  Aniruddha S. Gokhale,et al.  Supporting end-to-end quality of service properties in OMG data distribution service publish/subscribe middleware over wide area networks , 2013, J. Syst. Softw..

[12]  Peter R. Pietzuch,et al.  Hermes: a distributed event-based middleware architecture , 2002, Proceedings 22nd International Conference on Distributed Computing Systems Workshops.

[13]  Kaiwen Zhang,et al.  SDN-like: The Next Generation of Pub/Sub , 2013, ArXiv.

[14]  Roberto Baldoni,et al.  The Evolution of Publish/Subscribe Communication Systems , 2003, Future Directions in Distributed Computing.

[15]  Dale Skeen,et al.  The Information Bus: an architecture for extensible distributed systems , 1994, SOSP '93.

[16]  Aniruddha S. Gokhale,et al.  Bootstrapping Software Defined Network for flexible and dynamic control plane management , 2015, Proceedings of the 2015 1st IEEE Conference on Network Softwarization (NetSoft).

[17]  Roberto Beraldi,et al.  Efficient Publish/Subscribe Through a Self-Organizing Broker Overlay and its Application to SIENA , 2007, Comput. J..

[18]  Ming Li,et al.  A Scalable and Elastic Publish/Subscribe Service , 2011, 2011 IEEE International Parallel & Distributed Processing Symposium.

[19]  Fan Ye,et al.  Efficacy of techniques for responsiveness in a wide-area publish/subscribe system , 2010, Middleware Industrial Track '10.

[20]  David S. Rosenblum,et al.  Design and evaluation of a wide-area event notification service , 2001, TOCS.

[21]  Helge Parzyjegla,et al.  Self-organizing broker topologies for publish/subscribe systems , 2007, SAC '07.

[22]  Paolo Bellavista,et al.  Quality of Service in Wide Scale Publish—Subscribe Systems , 2014, IEEE Communications Surveys & Tutorials.

[23]  Pieter Hintjens,et al.  ZeroMQ: Messaging for Many Applications , 2013 .

[24]  Aniruddha S. Gokhale,et al.  Reliable publish/subscribe middleware for time-sensitive internet-scale applications , 2009, DEBS '09.

[25]  Mahadev Konar,et al.  ZooKeeper: Wait-free Coordination for Internet-scale Systems , 2010, USENIX ATC.