MERC: Match at Edge and Route intra--Cluster for Content-based Publish/Subscribe Systems

Despite suffering from inefficiency and flexibility limitations, the filter-based routing (FBR) algorithm is widely used in content-based publish/subscribe (pub/sub) systems. To address its limitations, we propose a dynamic destination-based routing algorithm called D-DBR, which decomposes pub/sub into two independent parts: Content-based matching and destination-based multicasting. D-DBR exhibits low event matching cost and high efficiency, flexibility, and robustness for event routing in small scale overlays. To boost scalability, we further complement D-DBR with a new routing algorithm called MERC. MERC divides the overlay into interconnected clusters and applies content-based and destination-based mechanisms to route events inter- and intra-cluster, respectively. We implemented all algorithms in the PADRES pub/sub system. Experimental results show that our algorithms outperform FBR in terms of improving event dissemination throughput by up to 700% and reducing the end-to-end latency by up to 55%.

[1]  Robbert van Renesse,et al.  Astrolabe: A robust and scalable technology for distributed system monitoring, management, and data mining , 2003, TOCS.

[2]  Giovanni Toffetti Carughi,et al.  Practical High-Throughput Content-Based Routing Using Unicast State and Probabilistic Encodings , 2009 .

[3]  Paolo Costa,et al.  Semi-Probabilistic Content-Based Publish-Subscribe , 2005, 25th IEEE International Conference on Distributed Computing Systems (ICDCS'05).

[4]  Stephen E. Deering,et al.  Multicast routing in datagram internetworks and extended LANs , 1990, TOCS.

[5]  Alejandro P. Buchmann,et al.  Filter Similarities in Content-Based Publish/Subscribe Systems , 2002, ARCS.

[6]  Sérgio Duarte,et al.  Routing algorithms for content-based publish/subscribe systems , 2010, IEEE Communications Surveys & Tutorials.

[7]  David S. Rosenblum,et al.  Design and evaluation of a wide-area event notification service , 2001, TOCS.

[8]  Felix C. Freiling,et al.  Evaluating advanced routing algorithms for content-based publish/subscribe systems , 2002, Proceedings. 10th IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunications Systems.

[9]  David D. Clark,et al.  Policy routing in Internet protocols , 1989, RFC.

[10]  Hans-Arno Jacobsen,et al.  A Unified Approach to Routing, Covering and Merging in Publish/Subscribe Systems Based on Modified Binary Decision Diagrams , 2005, 25th IEEE International Conference on Distributed Computing Systems (ICDCS'05).

[11]  Hans-Arno Jacobsen,et al.  The PADRES Distributed Publish/Subscribe System , 2005, FIW.

[12]  Hans-Arno Jacobsen,et al.  Foundations for Highly Available Content-Based Publish/Subscribe Overlays , 2011, 2011 31st International Conference on Distributed Computing Systems.

[13]  Philip S. Yu,et al.  Clustering algorithms for content-based publication-subscription systems , 2002, Proceedings 22nd International Conference on Distributed Computing Systems.

[14]  Emin Gün Sirer,et al.  Client behavior and feed characteristics of RSS, a publish-subscribe system for web micronews , 2005, IMC '05.

[15]  Joshua S. Auerbach,et al.  Exploiting IP Multicast in Content-Based Publish-Subscribe Systems , 2000, Middleware.

[16]  Todd L. Heberlein,et al.  Network intrusion detection , 1994, IEEE Network.

[17]  Hans-Arno Jacobsen,et al.  A distributed service-oriented architecture for business process execution , 2010, TWEB.

[18]  Hans-Arno Jacobsen,et al.  Safe Distribution and Parallel Execution of Data-Centric Workflows over the Publish/Subscribe Abstraction , 2015, IEEE Transactions on Knowledge and Data Engineering.

[19]  Antony I. T. Rowstron,et al.  Vigilante: End-to-end containment of Internet worm epidemics , 2006, TOCS.

[20]  Guruduth Banavar,et al.  An efficient multicast protocol for content-based publish-subscribe systems , 1999, Proceedings. 19th IEEE International Conference on Distributed Computing Systems (Cat. No.99CB37003).

[21]  Jaswinder Pal Singh,et al.  MEDYM: Match-Early with Dynamic Multicast for Content-Based Publish-Subscribe Networks , 2005, Middleware.

[22]  Alfonso Fuggetta,et al.  The JEDI Event-Based Infrastructure and Its Application to the Development of the OPSS WFMS , 2001, IEEE Trans. Software Eng..

[23]  Sheila A. McIlraith,et al.  Towards Planning the Transformation of Overlays , 2015, 2015 IEEE 35th International Conference on Distributed Computing Systems.

[24]  Donald F. Towsley,et al.  Channelization problem in large scale data dissemination , 2001, Proceedings Ninth International Conference on Network Protocols. ICNP 2001.

[25]  Nalini Venkatasubramanian,et al.  Dynamic Load Balancing for Cluster-Based Publish/Subscribe System , 2009, 2009 Ninth Annual International Symposium on Applications and the Internet.

[26]  Jörg Kienzle,et al.  A publish/subscribe network engine testbed for multiplayer games , 2014, Middleware.

[27]  Yoav Tock,et al.  Hierarchical Clustering of Message Flows in a Multicast Data Dissemination System , 2005, IASTED PDCS.

[28]  Nalini Venkatasubramanian,et al.  MICS: an efficient content space representation model for publish/subscribe systems , 2009, DEBS '09.

[29]  Paolo Costa,et al.  Introducing reliability in content-based publish-subscribe through epidemic algorithms , 2003, DEBS '03.

[30]  Hans-Arno Jacobsen,et al.  Adaptive Content-Based Routing in General Overlay Topologies , 2008, Middleware.

[31]  Jörg Kienzle,et al.  Publish/subscribe network designs for multiplayer games , 2014, Middleware.

[32]  Ellen W. Zegura,et al.  How to model an internetwork , 1996, Proceedings of IEEE INFOCOM '96. Conference on Computer Communications.

[33]  Michael B. Jones,et al.  Herald: achieving a global event notification service , 2001, Proceedings Eighth Workshop on Hot Topics in Operating Systems.

[34]  Mark Handley,et al.  Revisiting Content-Based Publish/Subscribe , 2006, 26th IEEE International Conference on Distributed Computing Systems Workshops (ICDCSW'06).