Adaptive Topology for Scalability and Immediacy in Distributed Publish/Subscribe Messaging

Publish/subscribe is a communication model for exchanging messages via a broker while providing loose coupling. So far, several studies have been conducted to address load concentration on the broker by forming distributed brokers. However, although they achieve higher throughput by load distribution among multiple brokers, these existing studies require an increased latency for message delivery. In this paper, we propose a novel method to construct and maintain an adaptive topology that features both scalability and immediacy in distributed publish/subscribe messaging. The proposed method is for topic-based publish/subscribe systems and uses a number of brokers to form an overlay network. Its topology changes dynamically to compose a subgraph for each topic in a single-hop or multi-hop manner according to the topic load (i.e., the number of clients). The experimental results show that compared to existing studies, the proposed method reduces the delivery path length, which is a principal factor that affects latency. Especially for low load topics, the reduction rate of the proposed method reaches values greater than 60%.

[1]  Jingyu Sun,et al.  Dissemination of edge-heavy data on heterogeneous MQTT brokers , 2017, 2017 IEEE 6th International Conference on Cloud Networking (CloudNet).

[2]  Anne-Marie Kermarrec,et al.  The many faces of publish/subscribe , 2003, CSUR.

[3]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[4]  Kazuyuki Shudo,et al.  FRT-Skip Graph: A Skip Graph-style structured overlay based on Flexible Routing Tables , 2016, 2016 IEEE Symposium on Computers and Communication (ISCC).

[5]  Kazuyuki Shudo,et al.  Interworking Layer of Distributed MQTT Brokers , 2019, IEICE Trans. Inf. Syst..

[6]  Peter Milligan,et al.  Range queries over skip tree graphs , 2008, Comput. Commun..

[7]  Kazuyuki Shudo,et al.  An Efficient Routing Method for Range Queries in Skip Graph , 2020, IEICE Trans. Inf. Syst..

[8]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[9]  James Aspnes,et al.  Skip graphs , 2003, SODA '03.

[10]  William Pugh,et al.  Skip Lists: A Probabilistic Alternative to Balanced Trees , 1989, WADS.

[11]  J. Manyika Big data: The next frontier for innovation, competition, and productivity , 2011 .

[12]  Michiharu Takemoto,et al.  SFB: a scalable method for handling range queries on Skip Graphs , 2015 .

[13]  Mark Handley,et al.  Application-Level Multicast Using Content-Addressable Networks , 2001, Networked Group Communication.

[14]  Michiharu Takemoto,et al.  Designing Overlay Networks for Handling Exhaust Data in a Distributed Topic-based Pub/Sub Architecture , 2015, J. Inf. Process..

[15]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.

[16]  Ben Y. Zhao,et al.  Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination , 2001, NOSSDAV '01.

[17]  Ron Kohavi,et al.  Online Experiments: Lessons Learned , 2007, Computer.

[18]  Kazuyuki Shudo,et al.  Flexible routing tables: Designing routing algorithms for overlays based on a total order on a routing table set , 2011, 2011 IEEE International Conference on Peer-to-Peer Computing.

[19]  Ben Y. Zhao,et al.  Tapestry: a resilient global-scale overlay for service deployment , 2004, IEEE Journal on Selected Areas in Communications.

[20]  Miguel Castro,et al.  Scribe: a large-scale and decentralized application-level multicast infrastructure , 2002, IEEE J. Sel. Areas Commun..