A Formal Framework for Complex Event Processing

Complex Event Processing (CEP) has emerged as the unifying field for technologies that require processing and correlating distributed data sources in real-time. CEP finds applications in diverse domains, which has resulted in a large number of proposals for expressing and processing complex events. However, existing CEP languages lack from a clear semantics, making them hard to understand and generalize. Moreover, there are no general techniques for evaluating CEP query languages with clear performance guarantees. In this paper we embark on the task of giving a rigorous and efficient framework to CEP. We propose a formal language for specifying complex events, called CEL, that contains the main features used in the literature and has a denotational and compositional semantics. We also formalize the so-called selection strategies, which had only been presented as by-design extensions to existing frameworks. With a well-defined semantics at hand, we discuss how to efficiently process complex events by evaluating CEL formulas with unary filters. We start by studying the syntactical properties of CEL and propose rewriting optimization techniques for simplifying the evaluation of formulas. Then, we introduce a formal computational model for CEP, called complex event automata (CEA), and study how to compile CEL formulas with unary filters into CEA. Furthermore, we provide efficient algorithms for evaluating CEA over event streams using constant time per event followed by constant-delay enumeration of the results. Finally, we gather the main results of this work to present an efficient and declarative framework for CEP.

[1]  Morris Sloman,et al.  GEM: a generalized event monitoring language for distributed systems , 1997, Distributed Syst. Eng..

[2]  Alejandro P. Buchmann,et al.  Complex Event Processing , 2009, it Inf. Technol..

[3]  Alessandro Margara,et al.  Processing flows of information: From data stream to complex event processing , 2012, CSUR.

[4]  Karin Ackermann Automation Production Systems And Computer Integrated Manufacturing , 2016 .

[5]  Luc Segoufin Automata and Logics for Words and Trees over an Infinite Alphabet , 2006, CSL.

[6]  Jennifer Widom,et al.  The CQL continuous query language: semantic foundations and query execution , 2006, The VLDB Journal.

[7]  Serge Abiteboul,et al.  Foundations of Databases: The Logical Level , 1995 .

[8]  Mariano Zelke,et al.  Algorithmic Techniques for Processing Data Streams , 2013, Data Exchange, Information, and Streams.

[9]  Alfred V. Aho,et al.  Algorithms for Finding Patterns in Strings , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[10]  Opher Etzion,et al.  Amit - the situation manager , 2003, The VLDB Journal.

[11]  Luc Segoufin,et al.  Enumerating with constant delay the answers to a query , 2013, ICDT '13.

[12]  Yanlei Diao,et al.  High-performance complex event processing over streams , 2006, SIGMOD Conference.

[13]  Juan Carlos Augusto,et al.  Two Approaches to Event Definition , 2002, DEXA.

[14]  David J. DeWitt,et al.  NiagaraCQ: a scalable continuous query system for Internet databases , 2000, SIGMOD '00.

[15]  Alexander Artikis,et al.  Logic-based event recognition , 2012, The Knowledge Engineering Review.

[16]  Chetan Gupta,et al.  E-Cube: multi-dimensional event sequence analysis using hierarchical pattern query sharing , 2011, SIGMOD '11.

[17]  Ugur Çetintemel,et al.  Plan-based complex event detection across distributed sources , 2008, Proc. VLDB Endow..

[18]  Peter R. Pietzuch,et al.  A Framework for Event Composition in Distributed Systems , 2003, Middleware.

[19]  Jeffrey F. Naughton,et al.  On Load Shedding in Complex Event Processing , 2013, ICDT.

[20]  Jonathan Goldstein,et al.  Consistent Streaming Through Time: A Vision for Event Stream Processing , 2006, CIDR.

[21]  Jayanthi Ranjan,et al.  Real time business intelligence in supply chain analytics , 2008, Inf. Manag. Comput. Secur..

[22]  Neil Immerman,et al.  On Supporting Kleene Closure over Event Streams , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[23]  Neil Immerman,et al.  Efficient pattern matching over event streams , 2008, SIGMOD Conference.

[24]  Alessandro Margara,et al.  Complex event processing with T-REX , 2012, J. Syst. Softw..

[25]  Alexander Artikis,et al.  An Event Calculus for Event Recognition , 2015, IEEE Transactions on Knowledge and Data Engineering.

[26]  Björn Lisper,et al.  A resource-efficient event algebra , 2010, Sci. Comput. Program..

[27]  Jacques Sakarovitch,et al.  Elements of Automata Theory , 2009 .

[28]  Lukasz Golab,et al.  Issues in data stream management , 2003, SGMD.

[29]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[30]  Alessandro Margara,et al.  TESLA: a formally defined event specification language , 2010, DEBS '10.

[31]  Sebastian Rudolph,et al.  A Rule-Based Language for Complex Event Processing and Reasoning , 2010, RR.

[32]  Margus Veanes Applications of Symbolic Finite Automata , 2013, CIAA.

[33]  Paola Mello,et al.  A Logic-Based, Reactive Calculus of Events , 2010, Fundam. Informaticae.

[34]  Johannes Gehrke,et al.  A General Algebra and Implementation for Monitoring Event Streams , 2005 .

[35]  Jennifer Widom,et al.  STREAM: the stanford stream data manager (demonstration description) , 2003, SIGMOD '03.

[36]  Jeffrey F. Naughton,et al.  On the complexity of privacy-preserving complex event processing , 2011, PODS.

[37]  Arnaud Durand,et al.  On Acyclic Conjunctive Queries and Constant Delay Enumeration , 2007, CSL.

[38]  Michael Stonebraker,et al.  Aurora: a data stream management system , 2003, SIGMOD '03.

[39]  Todd L. Heberlein,et al.  Network intrusion detection , 1994, IEEE Network.

[40]  Alessandro Margara,et al.  RACED: an adaptive middleware for complex event detection , 2009, ARM '09.

[41]  Johannes Gehrke,et al.  Towards Expressive Publish/Subscribe Systems , 2006, EDBT.

[42]  Matthias Weidlich,et al.  Complex Event Recognition Languages: Tutorial , 2017, DEBS.