Dynamic Load Balancing in Distributed Content-Based Publish/Subscribe

Distributed content-based publish/subscribe systems to date suffer from performance degradation and poor scalability caused by uneven load distributions typical in real-world applications. The reason for this shortcoming is due to the lack of a load balancing solution, which have rarely been studied in the context of publish/subscribe. This paper proposes a load balancing solution specific to distributed content-based publish/subscribe systems that is distributed, dynamic, adaptive, transparent, and accommodates heterogeneity. The solution consists of three key contributions: a load balancing framework, a novel load estimation algorithm, and three offload strategies. Experimental results show that the proposed load balancing solution is efficient with less than 1.5% overhead, effective with at least 91% load estimation accuracy, and capable of distributing all of the system's load originating from an edge point of the network.

[1]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[2]  Andrew P. Black,et al.  Fine-grained mobility in the Emerald system , 1987, TOCS.

[3]  Philip S. Yu,et al.  DNS dispatching algorithms with state estimators for scalable Web‐server clusters , 1999, World Wide Web.

[4]  Daniel M. Dias,et al.  A scalable and highly available web server , 1996, COMPCON '96. Technologies for the Information Superhighway Digest of Papers.

[5]  Anne-Marie Kermarrec,et al.  The many faces of publish/subscribe , 2003, CSUR.

[6]  Alejandro P. Buchmann,et al.  A peer-to-peer approach to content-based publish/subscribe , 2003, DEBS '03.

[7]  Michael J. Franklin,et al.  Efficient Filtering of XML Documents for Selective Dissemination of Information , 2000, VLDB.

[8]  Hong Va Leong,et al.  An extended CORBA event service with support for load balancing and fault-tolerance , 2000, Proceedings DOA'00. International Symposium on Distributed Objects and Applications.

[9]  Mukesh Singhal,et al.  Load distributing for locally distributed systems , 1992, Computer.

[10]  Gero Mühl,et al.  Generic Constraints for {Content-Based} Publish/Subscribe Systems , 2001 .

[11]  Thomas L. Casavant,et al.  A Taxonomy of Scheduling in General-Purpose Distributed Computing Systems , 1988, IEEE Trans. Software Eng..

[12]  Miguel Castro,et al.  SCRIBE: The Design of a Large-Scale Event Notification Infrastructure , 2001, Networked Group Communication.

[13]  Pascal Felber,et al.  A scalable protocol for content-based routing in overlay networks , 2003, Second IEEE International Symposium on Network Computing and Applications, 2003. NCA 2003..

[14]  Lars C. Wolf.  Proceedings of the 1st workshop on Network and system support for games , 2002 .

[15]  Gero Mühl.  Generic Constraints for Content-Based Publish/Subscribe , 2001, CoopIS.

[16]  Ken Shirriff,et al.  Building distributed process management on an object-oriented framework , 1997 .

[17]  Markus Aleksy,et al.  Design and Implementation of a Flexible Load Balancing Service for CORBA-based Applications , 2001 .

[18]  Jeff Kramer,et al.  Methodical Analysis of Adaptive Load Sharing Algorithms , 1992, IEEE Trans. Parallel Distributed Syst..

[19]  Guruduth Banavar,et al.  An efficient multicast protocol for content-based publish-subscribe systems , 1999, Proceedings. 19th IEEE International Conference on Distributed Computing Systems (Cat. No.99CB37003).

[20]  Bernd Freisleben,et al.  Load distribution in a CORBA environment , 1999, Proceedings of the International Symposium on Distributed Objects and Applications.

[21]  Jingwen Wang,et al.  Utopia: A load sharing facility for large, heterogeneous distributed computer systems , 1993, Softw. Pract. Exp..

[22]  Dennis Shasha,et al.  Filtering algorithms and implementation for very fast publish/subscribe systems , 2001, SIGMOD '01.

[23]  David S. Rosenblum,et al.  Design and evaluation of a wide-area event notification service , 2001, TOCS.

[24]  M. Litzkow.  REMOTE UNIX TURNING IDLE WORKSTATIONS INTO CYCLE SERVERS , 1992 .

[25]  Ben Y. Zhao,et al.  Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination , 2001, NOSSDAV '01.

[26]  David L. Black,et al.  An OSF/1 UNIX for Massively Parallel Multicomputers , 1993, USENIX Winter.

[27]  Francine Berman,et al.  Scheduling from the perspective of the application , 1996, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing.

[28]  Markus Lindermeier.  Load management for distributed object-oriented environments , 2000, Proceedings DOA'00. International Symposium on Distributed Objects and Applications.

[29]  Sasu Tarkoma,et al.  A Data Structure for Content-based Routing , 2005, EuroIMSA.

[30]  Srinivasan Seshan,et al.  Mercury: a scalable publish-subscribe system for internet games , 2002, NetGames '02.

[31]  Paolo Costa,et al.  Epidemic algorithms for reliable content-based publish-subscribe: an evaluation , 2004, 24th International Conference on Distributed Computing Systems, 2004. Proceedings..

[32]  Dennis Shasha,et al.  Efficient Matching for Web-Based Publish/Subscribe Systems , 2000, CoopIS.

[33]  Alfonso Fuggetta,et al.  The JEDI Event-Based Infrastructure and Its Application to the Development of the OPSS WFMS , 2001, IEEE Trans. Software Eng..

[34]  Peter R. Pietzuch,et al.  Hermes: a distributed event-based middleware architecture , 2002, Proceedings 22nd International Conference on Distributed Computing Systems Workshops.

[35]  Reinhard Lüling,et al.  A dynamic distributed load balancing algorithm with provable good performance , 1993, SPAA '93.

[36]  Dennis Shasha,et al.  WebFilter: A High-throughput XML-based Publish and Subscribe System , 2001, VLDB.

[37]  Douglas C. Schmidt,et al.  The Design of an Adaptive Middleware Load Balancing and Monitoring Service , 2003 .

[38]  Hans-Arno Jacobsen,et al.  Composite Subscriptions in Content-Based Publish/Subscribe Systems , 2005, Middleware.

[39]  Karsten Schwan,et al.  Opportunistic Overlays: Efficient Content Delivery in Mobile Ad Hoc Networks , 2005, Middleware.

[40]  Divyakant Agrawal,et al.  Meghdoot: Content-Based Publish/Subscribe over P2P Networks , 2004, Middleware.

[41]  Philip S. Yu,et al.  Dynamic Load Balancing on Web-Server Systems , 1999, IEEE Internet Comput..