Building Fault-Tolerant Overlays With Low Node Degrees for Topic-Based Publish/Subscribe

We present a new approach for designing reliable and scalable overlay networks to support topic-based pub/sub communication. We propose the <inline-formula><tex-math notation="LaTeX">${{\mathsf {MinAvg}}-{k}{\mathsf {TCO}}}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="sans-serif">MinAvg</mml:mi><mml:mo>-</mml:mo><mml:mi>k</mml:mi><mml:mi mathvariant="sans-serif">TCO</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq1-3080281.gif"/></alternatives></inline-formula> problem parameterized by <inline-formula><tex-math notation="LaTeX">${k}$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="chen-ieq2-3080281.gif"/></alternatives></inline-formula>: use the minimum number of edges to create a <italic><inline-formula><tex-math notation="LaTeX">${k}$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="chen-ieq3-3080281.gif"/></alternatives></inline-formula>-topic-connected overlay</italic> (<inline-formula><tex-math notation="LaTeX">${{k}TCO}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>k</mml:mi><mml:mi>T</mml:mi><mml:mi>C</mml:mi><mml:mi>O</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq4-3080281.gif"/></alternatives></inline-formula>) for pub/sub systems, i.e., for each topic, the sub-overlay induced by nodes interested in the topic is <inline-formula><tex-math notation="LaTeX">${k}$</tex-math><alternatives><mml:math><mml:mi>k</mml:mi></mml:math><inline-graphic xlink:href="chen-ieq5-3080281.gif"/></alternatives></inline-formula>-connected. We prove the NP-completeness of <inline-formula><tex-math notation="LaTeX">${{\mathsf {MinAvg}}-{k}{\mathsf {TCO}}}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="sans-serif">MinAvg</mml:mi><mml:mo>-</mml:mo><mml:mi>k</mml:mi><mml:mi mathvariant="sans-serif">TCO</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq6-3080281.gif"/></alternatives></inline-formula> and show a lower-bound for the hardness of its approximation. For <inline-formula><tex-math notation="LaTeX">${{\mathsf {MinAvg}}-{2}{\mathsf {TCO}}}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="sans-serif">MinAvg</mml:mi><mml:mo>-</mml:mo><mml:mn>2</mml:mn><mml:mi mathvariant="sans-serif">TCO</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq7-3080281.gif"/></alternatives></inline-formula>, we present GM2, the first polynomial-time algorithm with an approximation ratio. For <inline-formula><tex-math notation="LaTeX">${{\mathsf {MinAvg}}-{k}{\mathsf {TCO}}}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="sans-serif">MinAvg</mml:mi><mml:mo>-</mml:mo><mml:mi>k</mml:mi><mml:mi mathvariant="sans-serif">TCO</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq9-3080281.gif"/></alternatives></inline-formula>, where <inline-formula><tex-math notation="LaTeX">${k} \geq {2}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi>k</mml:mi><mml:mo>≥</mml:mo><mml:mn>2</mml:mn></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq10-3080281.gif"/></alternatives></inline-formula>, we propose HararyPT, a simple and efficient heuristic that aligns nodes across different sub-overlays. We experimentally demonstrate the scalability of GM2 and HararyPT with regards to overlay quality under representative pub/sub workloads. GM2 outputs <inline-formula><tex-math notation="LaTeX">${{2}TCO}$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>2</mml:mn><mml:mi>T</mml:mi><mml:mi>C</mml:mi><mml:mi>O</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq13-3080281.gif"/></alternatives></inline-formula> with an empirically insignificant increase in the average node degree, e.g., an increase by 4 in a 1000-node network, as compared to the baseline <inline-formula><tex-math notation="LaTeX">${{1}TCO}$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>1</mml:mn><mml:mi>T</mml:mi><mml:mi>C</mml:mi><mml:mi>O</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq14-3080281.gif"/></alternatives></inline-formula> produced by the best-known algorithm. Moreover, GM2 reduces the topic diameters by around 50 percent with respect to those in <inline-formula><tex-math notation="LaTeX">${{1}TCO}$</tex-math><alternatives><mml:math><mml:mrow><mml:mn>1</mml:mn><mml:mi>T</mml:mi><mml:mi>C</mml:mi><mml:mi>O</mml:mi></mml:mrow></mml:math><inline-graphic xlink:href="chen-ieq16-3080281.gif"/></alternatives></inline-formula>.

[1]  Yoav Tock,et al.  BeaConvey: Co-Design of Overlay and Routing for Topic-based Publish/Subscribe on Small-World Networks , 2018, DEBS.

[2]  Kaiwen Zhang,et al.  Incremental Topology Transformation for Publish/Subscribe Systems Using Integer Programming , 2017, 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS).

[3]  Yoav Tock,et al.  Overlay Design for Topic-Based Publish/Subscribe under Node Degree Constraints , 2016, 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS).

[4]  Hans-Arno Jacobsen,et al.  OMen: overlay mending for topic-based publish/subscribe systems under churn , 2016, DEBS.

[5]  Hans-Arno Jacobsen,et al.  Algorithms Based on Divide and Conquer for Topic-Based Publish/Subscribe Overlay Design , 2016, IEEE/ACM Transactions on Networking.

[6]  Yoav Tock,et al.  Design of Routing Protocols and Overlay Topologies for Topic-based Publish/Subscribe on Small-World Networks , 2015, Middleware Industry.

[7]  Yoav Tock,et al.  Weighted Overlay Design for Topic-Based Publish/Subscribe Systems on Geo-Distributed Data Centers , 2015, 2015 IEEE 35th International Conference on Distributed Computing Systems.

[8]  Kianoosh Mokhtarian,et al.  Minimum-Delay Multicast Algorithms for Mesh Overlays , 2015, IEEE/ACM Transactions on Networking.

[9]  James Aspnes,et al.  Network construction with subgraph connectivity constraints , 2015, J. Comb. Optim..

[10]  Hans-Arno Jacobsen,et al.  Planning the Transformation of Distributed Messaging Middlewares , 2014 .

[11]  Pascal Felber,et al.  Scaling Up Publish/Subscribe Overlays Using Interest Correlation for Link Sharing , 2013, IEEE Transactions on Parallel and Distributed Systems.

[12]  Hans-Arno Jacobsen,et al.  Brief announcement: constructing fault-tolerant overlay networks for topic-based publish/subscribe , 2013, PODC '13.

[13]  Vinay Setty,et al.  The Hidden Pub/Sub of Spotify , 2013, DEBS 2013.

[14]  Maarten van Steen,et al.  PolderCast: Fast, Robust, and Scalable Architecture for P2P Topic-Based Pub/Sub , 2012, Middleware.

[15]  Reza Sherafat Kazemzadeh,et al.  Opportunistic Multipath Forwarding in Content-Based Publish/Subscribe Overlays , 2012, Middleware.

[16]  Hans-Arno Jacobsen,et al.  A Generalized Algorithm for Publish/Subscribe Overlay Design and Its Fast Implementation , 2012, DISC.

[17]  Christopher Frost,et al.  Spanner: Google's Globally-Distributed Database , 2012, OSDI.

[18]  Hans-Arno Jacobsen,et al.  Scaling Construction of Low Fan-out Overlays for Topic-Based Publish/Subscribe Systems , 2011, 2011 31st International Conference on Distributed Computing Systems.

[19]  Andréa W. Richa,et al.  Minimum Maximum-Degree Publish-Subscribe Overlay Network Design , 2011, IEEE/ACM Trans. Netw..

[20]  Yoav Tock,et al.  Magnet: practical subscription clustering for Internet-scale publish/subscribe , 2010, DEBS '10.

[21]  Hans-Arno Jacobsen,et al.  Divide and Conquer Algorithms for Publish/Subscribe Overlay Design , 2010, 2010 IEEE 30th International Conference on Distributed Computing Systems.

[22]  Alec Wolman,et al.  Centrifuge: Integrated Lease Management and Partitioning for Cloud Services , 2010, NSDI.

[23]  Andréa W. Richa,et al.  Parameterized Maximum and Average Degree Approximation in Topic-Based Publish-Subscribe Overlay Network Design , 2009, 2010 IEEE 30th International Conference on Distributed Computing Systems.

[24]  Anne-Marie Kermarrec,et al.  Rappel: Exploiting interest and network locality to improve fairness in publish-subscribe systems , 2009, Comput. Networks.

[25]  Hans-Arno Jacobsen,et al.  Adaptive Content-Based Routing in General Overlay Topologies , 2008, Middleware.

[26]  Hans-Arno Jacobsen,et al.  PNUTS: Yahoo!'s hosted data serving platform , 2008, Proc. VLDB Endow..

[27]  Nick McKeown,et al.  OpenFlow: enabling innovation in campus networks , 2008, CCRV.

[28]  Wilson C. Hsieh,et al.  Bigtable: A Distributed Storage System for Structured Data , 2006, TOCS.

[29]  Yoav Tock,et al.  Constructing scalable overlays for pub-sub with many topics , 2007, PODC '07.

[30]  Roberto Beraldi,et al.  Efficient Publish/Subscribe Through a Self-Organizing Broker Overlay and its Application to SIENA , 2007, Comput. J..

[31]  Roberto Beraldi,et al.  TERA: topic-based event routing for peer-to-peer architectures , 2007, DEBS '07.

[32]  Yoav Tock,et al.  SpiderCast: a scalable interest-aware overlay for topic-based pub/sub communication , 2007, DEBS '07.

[33]  Emin Gün Sirer,et al.  Client behavior and feed characteristics of RSS, a publish-subscribe system for web micronews , 2005, IMC '05.

[34]  Pascal Felber,et al.  Semantic Peer-to-Peer Overlays for Publish/Subscribe Networks , 2005, Euro-Par.

[35]  Yoav Tock,et al.  Hierarchical Clustering of Message Flows in a Multicast Data Dissemination System , 2005, IASTED PDCS.

[36]  Hans-Arno Jacobsen,et al.  Small Scale Peer-to-Peer Publish/Subscribe , 2005, P2PKM.

[37]  Idit Keidar,et al.  Araneola: a scalable reliable multicast system for dynamic environments , 2004, Third IEEE International Symposium on Network Computing and Applications, 2004. (NCA 2004). Proceedings..

[38]  Patrick Th. Eugster,et al.  Data-aware multicast , 2004, International Conference on Dependable Systems and Networks, 2004.

[39]  Hans-Arno Jacobsen,et al.  Building Content-Based Publish/Subscribe Systems with Distributed Hash Tables , 2003, DBISP2P.

[40]  Krishna P. Gummadi,et al.  The impact of DHT routing geometry on resilience and proximity , 2003, SIGCOMM '03.

[41]  Miguel Castro,et al.  Scribe: a large-scale and decentralized application-level multicast infrastructure , 2002, IEEE J. Sel. Areas Commun..

[42]  David R. Karger,et al.  Chord: A scalable peer-to-peer lookup service for internet applications , 2001, SIGCOMM '01.

[43]  Dennis Clark,et al.  The Prime Number Theorem , 2002 .

[44]  Keith Marzullo,et al.  Gossip versus Deterministically Constrained Flooding on Small Networks , 2000, DISC.

[45]  Jon M. Kleinberg,et al.  The small-world phenomenon: an algorithmic perspective , 2000, STOC '00.

[46]  Kathryn Fraughnaugh,et al.  Introduction to graph theory , 1997, Networks.