Adjusting Matching Algorithm to Adapt to Workload Fluctuations in Content-based Publish/Subscribe Systems

When facing fluctuating workloads, can the performance of matching algorithms in a content-based publish/subscribe system be adjusted to adapt to the workloads? In this paper, we explore the idea of endowing matching algorithms with adaptability. The prerequisite for adaptability is to enable the matching algorithm to possess the ability to dynamically and quantitatively adjust its performance. We propose PSAM, a Predicate-Skipping Adjustment Mechanism that realizes dynamic performance adjustment by smoothly switching between exact matching and approximate matching, following the strategy of trading off matching precision in favor of matching speed. The PSAM mechanism is integrated into an existing matching algorithm, resulting in a performance-adjustable matching algorithm called Ada-Rein. To collaborate with Ada-Rein, we design PADA, a Performance Adjustment Decision Algorithm that is able to make proper performance adjustment plans in the presence of fluctuating workloads. The effectiveness of Ada-Rein and PADA is evaluated through a series of experiments based on both synthetic data and real-world stock traces. Experiment results show that adjusting the performance of Ada-Rein at the price of a small false positive rate, less than 0.1%, can shorten event latency by almost 2.1 times, which well demonstrates the feasibility of our exploratory idea.

[1]  R. L. Thorndike.  Who belongs in the family? , 1953 .

[2]  TanKian-Lee,et al.  An efficient publish/subscribe index for e-commerce databases , 2014, VLDB 2014.

[3]  Marcos K. Aguilera,et al.  Matching events in a content-based subscription system , 1999, PODC '99.

[4]  Minglu Li,et al.  H-Tree: An Efficient Index Structurefor Event Matching in Content-BasedPublish/Subscribe Systems , 2015, IEEE Transactions on Parallel and Distributed Systems.

[5]  Thomas S. Heinze,et al.  Elastic Scaling of a High-Throughput Content-Based Publish/Subscribe Engine , 2014, 2014 IEEE 34th International Conference on Distributed Computing Systems.

[6]  Marcus Fontoura,et al.  Top-k Publish-Subscribe for Social Annotation of News , 2013, Proc. VLDB Endow..

[7]  Yijie Wang,et al.  A cloud‐assisted publish/subscribe service for time‐critical dissemination of bulk content , 2017, Concurr. Comput. Pract. Exp..

[8]  Yun Li,et al.  PID control system analysis, design, and technology , 2005, IEEE Transactions on Control Systems Technology.

[9]  Jörg Kienzle,et al.  Dynamoth: A Scalable Pub/Sub Middleware for Latency-Constrained Applications in the Cloud , 2015, 2015 IEEE 35th International Conference on Distributed Computing Systems.

[10]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[11]  Alexander L. Wolf,et al.  Forwarding in a content-based network , 2003, SIGCOMM '03.

[12]  Jie Wu,et al.  Towards Approximate Event Processing in a Large-Scale Content-Based Network , 2011, 2011 31st International Conference on Distributed Computing Systems.

[13]  Minglu Li,et al.  REIN: A fast event matching approach for content-based publish/subscribe systems , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[14]  Hans-Arno Jacobsen,et al.  Load Balancing Content-Based Publish/Subscribe Systems , 2010, TOCS.

[15]  G. Zipf,et al.  Relative Frequency as a Determinant of Phonetic Change , 1930 .

[16]  Hans-Arno Jacobsen,et al.  OMen: overlay mending for topic-based publish/subscribe systems under churn , 2016, DEBS.

[17]  Anne-Marie Kermarrec,et al.  The many faces of publish/subscribe , 2003, CSUR.

[18]  Li Fan,et al.  Web caching and Zipf-like distributions: evidence and implications , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[19]  Nalini Venkatasubramanian,et al.  MICS: an efficient content space representation model for publish/subscribe systems , 2009, DEBS '09.

[20]  Sergey Brin,et al.  The Anatomy of a Large-Scale Hypertextual Web Search Engine , 1998, Comput. Networks.

[21]  Sergei Vassilvitskii,et al.  Indexing Boolean Expressions , 2009, Proc. VLDB Endow..

[22]  Hans-Arno Jacobsen,et al.  BE-tree: an index structure to efficiently match boolean expressions over high-dimensional discrete space , 2011, SIGMOD '11.

[23]  Yijie Wang,et al.  A General Scalable and Elastic Content-Based Publish/Subscribe Service , 2015, IEEE Transactions on Parallel and Distributed Systems.

[24]  Yuanan Liu,et al.  GEM: An analytic geometrical approach to fast event matching for multi-dimensional content-based publish/subscribe services , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[25]  Yuanan Liu,et al.  DEXIN: A fast content-based multi-attribute event matching algorithm using dynamic exclusive and inclusive methods , 2017, Future Gener. Comput. Syst..