A Large-scale and Decentralized Infrastructure for Content-based Publish/Subscribe Services

Publish/subscribe model has become a prevalent paradigm for building distributed event delivering systems. Content-based publish/subscribe allows high expresses in subscriptions and thus is more appropriate for content dissemination. However, the scalability has remained a challenge in the design of distributed content-based publish/subscribe systems due to the expensive matching and delivering cost of content-based events. In this paper we propose HyperSub, an infrastructure built on top of distributed hash table (DHT) for efficient content-based data distribution. Based on efficient subscription installation, event publication and event delivery mechanisms, Hyper- Sub can provide a scalable platform to simultaneously support any numbers of pub/sub schemes with different number of attributes. There are three key features in the design of HyperSub: (1) a locality-preserving hashing mechanism which partitions the content space into content zones. Subscriptions and events are mapped to the content zones for efficiently matching; (2) an efficient event delivery algorithm which exploits the embedded trees in the underlying DHT to deliver events to the corresponding subscribers; (3) light-weighted load balancing mechanisms to adjust the load among peers and ensure that no peer is unduly loaded.

[1]  Wednesday September,et al.  2007 International Conference on Parallel Processing , 2007 .

[2]  Ben Y. Zhao,et al.  Bayeux: an architecture for scalable and fault-tolerant wide-area data dissemination , 2001, NOSSDAV '01.

[3]  G. Perng Providing content-based services in a peer-to-peer environment , 2004, ICSE 2004.

[4]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[5]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[6]  Peter Triantafillou,et al.  PastryStrings: A Comprehensive Content-Based Publish/Subscribe DHT Network , 2006, 26th IEEE International Conference on Distributed Computing Systems (ICDCS'06).

[7]  Miguel Castro,et al.  SCRIBE: The Design of a Large-Scale Event Notification Infrastructure , 2001, Networked Group Communication.

[8]  Antony I. T. Rowstron,et al.  Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems , 2001, Middleware.

[9]  Hans-Arno Jacobsen,et al.  Building Content-Based Publish/Subscribe Systems with Distributed Hash Tables , 2003, DBISP2P.

[10]  Y. Choi HOMED: a peer-to-peer overlay architecture for large-scale content-based publish/subscribe system , 2004, ICSE 2004.

[11]  Peter Triantafillou Content-based publish-subscribe over structured P2P networks , 2004, ICSE 2004.

[12]  Yiming Hu,et al.  Ferry: an architecture for content-based publish/subscribe services on P2P networks , 2005, 2005 International Conference on Parallel Processing (ICPP'05).

[13]  Dennis Shasha,et al.  Filtering algorithms and implementation for very fast publish/subscribe systems , 2001, SIGMOD '01.

[14]  Kenneth P. Birman,et al.  The process group approach to reliable distributed computing , 1992, CACM.

[15]  Robert Tappan Morris,et al.  Designing a DHT for Low Latency and High Throughput , 2004, NSDI.

[16]  Dale Skeen,et al.  The Information Bus: an architecture for extensible distributed systems , 1994, SOSP '93.

[17]  Krishna P. Gummadi,et al.  King: estimating latency between arbitrary internet end hosts , 2002, IMW '02.

[18]  David S. Rosenblum,et al.  Design and evaluation of a wide-area event notification service , 2001, TOCS.

[19]  Ben Y. Zhao,et al.  An Infrastructure for Fault-tolerant Wide-area Location and Routing , 2001 .

[20]  David R. Karger,et al.  Chord: a scalable peer-to-peer lookup protocol for internet applications , 2003, TNET.

[21]  Alejandro P. Buchmann,et al.  A peer-to-peer approach to content-based publish/subscribe , 2003, DEBS '03.

[22]  Guruduth Banavar,et al.  An efficient multicast protocol for content-based publish-subscribe systems , 1999, Proceedings. 19th IEEE International Conference on Distributed Computing Systems (Cat. No.99CB37003).

[23]  Peter R. Pietzuch,et al.  Peer-to-peer overlay broker networks in an event-based middleware , 2003, DEBS '03.

[24]  Divyakant Agrawal,et al.  Meghdoot: Content-Based Publish/Subscribe over P2P Networks , 2004, Middleware.

[25]  Mark Handley,et al.  A scalable content-addressable network , 2001, SIGCOMM '01.