Managing Measurement and Occurrence Uncertainty in Complex Event Processing Systems

Complex event processing (CEP) is a powerful technology for analyzing streams of real-time events, coming from different sources, and for extracting conclusions from them. In many situations, these events are not free from uncertainty, due to either unreliable data sources and networks, measurement uncertainty, or inability to determine whether an event has actually happened or not. This paper presents a proposal for incorporating and managing different kinds of uncertainty that may happen in both events and rules of the CEP systems. We provide a library that enables the representation and propagation of uncertain values, which can be efficiently integrated with the existing CEP languages and engines to deal with uncertainty, and we show how the treatment of uncertainty can be smoothly added to two of them: Esper and Apache Flink. Five applications coming from various domains serve to evaluate the proposal and to analyze its performance and accuracy. The results show that the overhead introduced by the treatment of uncertainty is not high and good precision and recall are achieved.

[1]  Inmaculada Medina-Bulo,et al.  MEdit4CEP: A model-driven solution for real-time decision making in SOA 2.0 , 2015, Knowl. Based Syst..

[2]  Xin Li,et al.  Complex Event Processing over Uncertain Data Streams , 2010, 2010 International Conference on P2P, Parallel, Grid, Cloud and Internet Computing.

[3]  Antonio Vallecillo,et al.  Expressing Measurement Uncertainty in OCL/UML Datatypes , 2018, ECMFA.

[4]  Alexander Artikis,et al.  Probabilistic Complex Event Recognition , 2017, ACM Comput. Surv..

[5]  Avigdor Gal,et al.  Complex event processing over uncertain data , 2008, DEBS.

[6]  Dawei Jin,et al.  Real-time Public Mood Tracking of Chinese Microblog Streams with Complex Event Processing , 2017, IEEE Access.

[7]  Juan Boubeta-Puig,et al.  COLLECT: COLLaborativE ConText-aware service oriented architecture for intelligent decision-making in the Internet of Things , 2017, Expert Syst. Appl..

[8]  Johannes Gehrke,et al.  Cayuga: a high-performance event processing engine , 2007, SIGMOD '07.

[9]  Matteo Pradella,et al.  Efficient analysis of event processing applications , 2015, DEBS.

[10]  Christopher Ré,et al.  Event queries on correlated probabilistic streams , 2008, SIGMOD Conference.

[11]  Antonio Vallecillo,et al.  Static Analysis of Complex Event Processing Programs , 2018, MoDELS.

[12]  Antonio Vallecillo,et al.  Formalizing Complex Event Processing Systems in Maude , 2018, IEEE Access.

[13]  Yushun Fan,et al.  Complex event processing in enterprise information systems based on RFID , 2007, Enterp. Inf. Syst..

[14]  Xiaoming Zhang,et al.  Complex Event Processing over distributed probabilistic event streams , 2012, FSKD.

[15]  Antonio Vallecillo,et al.  Managing Uncertain Complex Events in Web of Things Applications , 2018, ICWE.

[16]  Johannes Gehrke,et al.  Towards Expressive Publish/Subscribe Systems , 2006, EDBT.

[17]  D. Luckham.  Event Processing for Business: Organizing the Real-Time Enterprise , 2011 .

[18]  Neil Immerman,et al.  On complexity and optimization of expensive queries in complex event processing , 2014, SIGMOD Conference.

[19]  Minos N. Garofalakis,et al.  Issues in complex event processing: Status and prospects in the Big Data era , 2017, J. Syst. Softw..

[20]  Neil Immerman,et al.  Efficient pattern matching over event streams , 2008, SIGMOD Conference.

[21]  Dan Olteanu,et al.  $${10^{(10^{6})}}$$ worlds and beyond: efficient representation and processing of incomplete information , 2006, 2007 IEEE 23rd International Conference on Data Engineering.

[22]  Zoran Budimac,et al.  Redefining Software Quality Metrics to XML Schema Needs , 2013, SQAMIA.

[23]  Lieven Eeckhout,et al.  Statistically rigorous java performance evaluation , 2007, OOPSLA.

[24]  Hongjun Lu,et al.  A false negative approach to mining frequent itemsets from high speed transactional data streams , 2006, Inf. Sci..

[25]  Samuel Greengard,et al.  The Internet of Things , 2015 .

[26]  Sebastian Rudolph,et al.  Stream reasoning and complex event processing in ETALIS , 2012, Semantic Web.

[27]  Emanuele Della Valle,et al.  Defining the execution semantics of stream processing engines , 2017, Journal of Big Data.

[28]  David Luckham,et al.  The power of events - an introduction to complex event processing in distributed enterprise systems , 2002, RuleML.

[29]  B. D. Finetti,et al.  Theory of Probability: A Critical Introductory Treatment , 2017 .

[30]  Alessandro Margara,et al.  Processing flows of information: From data stream to complex event processing , 2012, CSUR.

[31]  Bran Selic,et al.  Understanding Uncertainty in Cyber-Physical Systems: A Conceptual Model , 2016, ECMFA.

[32]  Jianzhong Qiao,et al.  Complex Event Detection in Probabilistic Stream , 2010, 2010 12th International Asia-Pacific Web Conference.

[33]  Qian Yang,et al.  Building complex event processing capability for intelligent environmental monitoring , 2019, Environ. Model. Softw..

[34]  Kathleen V. Diegert,et al.  Error and uncertainty in modeling and simulation , 2002, Reliab. Eng. Syst. Saf..

[35]  Alessandro Margara,et al.  Complex event processing with T-REX , 2012, J. Syst. Softw..

[36]  Nikos Pelekis,et al.  Online event recognition from moving vessel trajectories , 2016, GeoInformatica.

[37]  Giordano Tamburrelli,et al.  Introducing uncertainty in complex event processing: model, implementation, and validation , 2014, Computing.

[38]  Johannes Gehrke,et al.  Cayuga: A General Purpose Event Monitoring System , 2007, CIDR.

[39]  Juan Boubeta-Puig,et al.  Complex Event Processing Modeling by Prioritized Colored Petri Nets , 2016, IEEE Access.

[40]  B. Kosko.  Fuzziness vs. probability , 1990 .

[41]  Craig Chambers,et al.  The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing , 2015, Proc. VLDB Endow..

[42]  Krysia Broda,et al.  SAGE: A Logical Agent-Based Environment Monitoring and Control System , 2009, AmI.

[43]  Opher Etzion,et al.  Event processing under uncertainty , 2012, DEBS.

[44]  Neil Immerman,et al.  Recognizing patterns in streams with imprecise timestamps , 2013, Inf. Syst..

[45]  Earl T. Barr,et al.  Uncertainty, risk, and information value in software requirements and architecture , 2014, ICSE.