Brief announcement: constructing fault-tolerant overlay networks for topic-based publish/subscribe

We incorporate fault tolerance in designing reliable and scalable overlay networks to support topic-based pub/sub communication. We propose the MinAvg- kTCO problem parameterized by k: use the minimum number of edges to create a k-topic-connected overlay (kTCO) for pub/sub systems, i.e., for each topic the sub-overlay induced by nodes interested in the topic is k-connected. We prove the NP-completeness of MinAvg-kTCO and show a lower-bound for the hardness of its approximation. With regard to MinAvg-2TCO, we present GM2, the first polynomial time algorithm with an approximation ratio. With regards to MinAvg-kTCO, where k ≥ 2, we propose a simple and efficient heuristic algorithm, namely HararyPT, that aligns nodes across different sub-overlays. We experimentally demonstrate the scalability of GM2 and HararyPT under representative pub/sub workloads.

[1]  Fabrizio Grandoni,et al.  Fast Low Degree Connectivity of Ad-Hoc Networks Via Percolation , 2007, ESA.

[2]  Alec Wolman,et al.  Centrifuge: Integrated Lease Management and Partitioning for Cloud Services , 2010, NSDI.

[3]  Hans-Arno Jacobsen,et al.  Scaling Construction of Low Fan-out Overlays for Topic-Based Publish/Subscribe Systems , 2011, 2011 31st International Conference on Distributed Computing Systems.

[4]  Hans-Arno Jacobsen,et al.  Efficient event processing through reconfigurable hardware for algorithmic trading , 2010, Proc. VLDB Endow..

[5]  Yoav Tock,et al.  SpiderCast: a scalable interest-aware overlay for topic-based pub/sub communication , 2007, DEBS '07.

[6]  Mohit Singh,et al.  Survivable network design with degree or order constraints , 2007, STOC '07.

[7]  Yoav Tock,et al.  Hierarchical Clustering of Message Flows in a Multicast Data Dissemination System , 2005, IASTED PDCS.

[8]  Frank Dürr,et al.  The power of software-defined networking: line-rate content-based routing using OpenFlow , 2012, MW4NG '12.

[9]  Andréa W. Richa,et al.  Minimum Maximum Degree Publish-Subscribe Overlay Network Design , 2009, IEEE INFOCOM 2009.

[10]  James Aspnes,et al.  Inferring Social Networks from Outbreaks , 2010, ALT.

[11]  Bernard Harris,et al.  Graph theory and its applications , 1970 .

[12]  Roberto Beraldi,et al.  Efficient Publish/Subscribe Through a Self-Organizing Broker Overlay and its Application to SIENA , 2007, Comput. J..

[13]  Hans-Arno Jacobsen,et al.  A Generalized Algorithm for Publish/Subscribe Overlay Design and Its Fast Implementation , 2012, DISC.

[14]  Hans-Arno Jacobsen,et al.  PNUTS: Yahoo!'s hosted data serving platform , 2008, Proc. VLDB Endow..

[15]  Andréa W. Richa,et al.  Parameterized Maximum and Average Degree Approximation in Topic-Based Publish-Subscribe Overlay Network Design , 2009, 2010 IEEE 30th International Conference on Distributed Computing Systems.

[16]  Pascal Felber,et al.  Semantic Peer-to-Peer Overlays for Publish/Subscribe Networks , 2005, Euro-Par.

[17]  Emin Gün Sirer,et al.  Client behavior and feed characteristics of RSS, a publish-subscribe system for web micronews , 2005, IMC '05.

[18]  Helge Parzyjegla,et al.  Self-organizing broker topologies for publish/subscribe systems , 2007, SAC '07.

[19]  Maarten van Steen,et al.  PolderCast: Fast, Robust, and Scalable Architecture for P2P Topic-Based Pub/Sub , 2012, Middleware.

[20]  Yoav Tock,et al.  Constructing scalable overlays for pub-sub with many topics , 2007, PODC '07.

[21]  Ming Li,et al.  A Scalable and Elastic Publish/Subscribe Service , 2011, 2011 IEEE International Parallel & Distributed Processing Symposium.

[22]  Tarek F. Abdelzaher,et al.  GreenGPS: a participatory sensing fuel-efficient maps application , 2010, MobiSys '10.

[23]  Hans-Arno Jacobsen,et al.  A distributed service-oriented architecture for business process execution , 2010, TWEB.

[24]  Roberto Beraldi,et al.  TERA: topic-based event routing for peer-to-peer architectures , 2007, DEBS '07.

[25]  Miguel Castro,et al.  Scribe: a large-scale and decentralized application-level multicast infrastructure , 2002, IEEE J. Sel. Areas Commun..

[26]  Yoav Tock,et al.  Magnet: practical subscription clustering for Internet-scale publish/subscribe , 2010, DEBS '10.

[27]  David R. Karger,et al.  Analysis of the evolution of peer-to-peer systems , 2002, PODC '02.

[28]  Hans-Arno Jacobsen,et al.  Divide and Conquer Algorithms for Publish/Subscribe Overlay Design , 2010, 2010 IEEE 30th International Conference on Distributed Computing Systems.

[29]  Kathryn Fraughnaugh,et al.  Introduction to graph theory , 1973, Mathematical Gazette.

[30]  Amy L. Murphy,et al.  Efficient content-based event dispatching in the presence of topological reconfiguration , 2003, 23rd International Conference on Distributed Computing Systems, 2003. Proceedings..

[31]  Dennis Clark,et al.  The Prime Number Theorem , 2002 .

[32]  Vinny Cahill,et al.  STEAM: event-based middleware for wireless ad hoc networks , 2002, Proceedings 22nd International Conference on Distributed Computing Systems Workshops.

[33]  Yang Liu,et al.  Be conservative: enhancing failure diagnosis with proactive logging , 2012, OSDI 2012.

[34]  Patrick Th. Eugster,et al.  Data-aware multicast , 2004, International Conference on Dependable Systems and Networks, 2004.

[35]  José Pereira,et al.  StAN: exploiting shared interests without disclosing them in gossip-based publish/subscribe , 2010, IPTPS.