Adaptive Online First-Order Monitoring

Online first-order monitoring is the task of detecting temporal patterns in streams of events carrying data. Considerable research has been devoted to scaling up monitoring using parallelization by partitioning events based on their data values and processing the partitions concurrently. To be effective, partitioning must account for the event stream’s statistics, e.g., the relative event frequencies, and these statistics may change rapidly. We develop the first parallel online first-order monitor capable of adapting to such changes. A central problem we solve is how to manage and exchange states between the parallel executing monitors. To this end, we develop state exchange operations and prove their correctness. Moreover, we extend the implementation of the MonPoly monitoring tool with these operations, thereby supporting parallel adaptive monitoring, and show empirically that adaptation can yield up to a tenfold improvement in run-time.

[1]  David Basin,et al.  A Formally Verified Monitor for Metric First-Order Temporal Logic , 2019, RV.

[2]  Sylvain Hallé,et al.  MapReduce for parallel trace validation of LTL properties , 2015, Journal of Cloud Computing.

[3]  Grigore Rosu,et al.  Semantics and Algorithms for Parametric Monitoring , 2011, Log. Methods Comput. Sci..

[4]  Vasiliki Kalavri,et al.  Megaphone: Latency-conscious state migration for distributed streaming dataflows , 2018, Proc. VLDB Endow..

[5]  Felix Naumann,et al.  The Stratosphere platform for big data analytics , 2014, The VLDB Journal.

[6]  Christoph Koch,et al.  Squall: Scalable Real-time Analytics , 2016, Proc. VLDB Endow..

[7]  Carlo Ghezzi,et al.  Efficient Large-Scale Trace Checking Using MapReduce , 2015, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[8]  David A. Basin,et al.  Scalable online first-order monitoring , 2018, International Journal on Software Tools for Technology Transfer.

[9]  Assaf Schuster,et al.  Efficient Adaptive Detection of Complex Event Patterns , 2018, Proc. VLDB Endow..

[10]  David E. Rydeheard,et al.  From First-order Temporal Logic to Parametric Trace Slicing , 2015, RV.

[11]  Felix Klaedtke,et al.  The MonPoly Monitoring Tool , 2017, RV-CuBES.

[12]  Kurt Rothermel,et al.  Minimizing Communication Overhead in Window-Based Parallel Complex Event Processing , 2017, DEBS.

[13]  Felix Klaedtke,et al.  Monitoring Metric First-Order Temporal Properties , 2015, J. ACM.

[14]  Joseph M. Hellerstein,et al.  Flux: an adaptive partitioning operator for continuous query systems , 2003, Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405).

[15]  Mathieu Klein,et al.  MapReduce for Parallel Trace Validation of LTL Properties , 2012, RV.

[16]  Christoph Koch,et al.  Scalable and Adaptive Online Joins , 2014, Proc. VLDB Endow..

[17]  Vasiliki Kalavri,et al.  Three steps is all you need: fast, accurate, automatic scaling decisions for distributed streaming dataflows , 2018, OSDI.

[18]  Rajeev Alur,et al.  Modular quantitative monitoring , 2019, Proc. ACM Program. Lang..

[19]  Jeffrey D. Ullman,et al.  Optimizing Multiway Joins in a Map-Reduce Environment , 2011, IEEE Transactions on Knowledge and Data Engineering.

[20]  Carlo Ghezzi,et al.  Trace Checking of Metric Temporal Logic with Aggregating Modalities Using MapReduce , 2014, SEFM.

[21]  Felix Klaedtke,et al.  Scalable offline monitoring of temporal specifications , 2016, Formal Methods in System Design.

[22]  M. Abadi,et al.  Naiad: a timely dataflow system , 2013, SOSP.

[23]  Roberto Baldoni,et al.  Adaptive online scheduling in storm , 2013, DEBS.

[24]  Felix Klaedtke,et al.  Greedily computing associative aggregations on sliding windows , 2015, Inf. Process. Lett..