EasyFlinkCEP: Big Event Data Analytics for Everyone

FlinkCEP is the Complex Event Processing (CEP) API of the Flink Big Data platform. The high expressive power of the language of FlinkCEP comes at the cost of cumbersome parameterization of the queried patterns, acting as a barrier for FlinkCEP's adoption. Moreover, properly configuring a FlinkCEP program to run over a computer cluster requires advanced skills on modern hardware administration which non-expert programmers do not possess. In this work (i) we build a novel, logical CEP operator that receives CEP pattern queries in the form of extended regular expressions and seamlessly re-writes them to FlinkCEP programs, (ii) we build a CEP Optimizer that automatically decides good job configurations for these FlinkCEP programs. We also present an experimental evaluation which demonstrates the significant benefits of our approach.

[1]  Antonios Deligiannakis,et al.  Towards creating a generalized complex event processing operator using FlinkCEP: architecture & benchmark , 2021, DEBS.

[2]  Antonios Deligiannakis,et al.  Real-time processing of geo-distributed financial data , 2021, DEBS.

[3]  Alkis Simitsis,et al.  Extreme-Scale Interactive Cross-Platform Streaming Analytics - The INFORE Approach , 2021, SEA-Data@VLDB.

[4]  Minos N. Garofalakis,et al.  INforE: Interactive Cross-platform Analytics for Everyone , 2020, CIKM.

[5]  Dimitris Zissis,et al.  Experimental Comparison of Complex Event Processing Systems in the Maritime Domain , 2020, 2020 21st IEEE International Conference on Mobile Data Management (MDM).

[6]  Stijn Vansummeren,et al.  On the Expressiveness of Languages for Complex Event Recognition , 2020, ICDT.

[7]  Alexander Artikis,et al.  Complex event recognition in the Big Data era: a survey , 2019, The VLDB Journal.

[8]  Cyril Ray,et al.  Composite Event Recognition for Maritime Monitoring , 2019, DEBS.

[9]  Alexander Artikis,et al.  Wayeb: a Tool for Complex Event Forecasting , 2018, LPAR.

[10]  Ruben Mayer,et al.  Window-based data parallelization in complex event processing , 2018 .

[11]  Kurt Rothermel,et al.  SPECTRE: supporting consumption policies in window-based parallel complex event processing , 2017, Middleware.

[12]  Alexander Artikis,et al.  Complex event recognition in the Big Data era: a survey , 2017, The VLDB Journal.

[13]  Matthias Weidlich,et al.  Complex Event Recognition Languages: Tutorial , 2017, DEBS.

[14]  Kurt Rothermel,et al.  Minimizing Communication Overhead in Window-Based Parallel Complex Event Processing , 2017, DEBS.

[15]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[16]  Minos N. Garofalakis,et al.  Issues in Complex Event Processing Systems , 2015, 2015 IEEE Trustcom/BigDataSE/ISPA.

[17]  Srinath Perera,et al.  Solution patterns for realtime streaming analytics , 2015, DEBS.

[18]  Neil Immerman,et al.  On complexity and optimization of expensive queries in complex event processing , 2014, SIGMOD Conference.

[19]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[20]  Samuel Madden,et al.  ZStream: a cost-based query processor for adaptively detecting composite events , 2009, SIGMOD Conference.

[21]  Neil Immerman,et al.  Efficient pattern matching over event streams , 2008, SIGMOD Conference.

[22]  Asaf Adi,et al.  Complex Event Processing for Financial Services , 2006, 2006 IEEE Services Computing Workshops.

[23]  Timos K. Sellis,et al.  Window Specification over Data Streams , 2006, EDBT Workshops.

[24]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .