Complex event processing over uncertain events: Techniques, challenges, and future directions

In the recent years, there is a massive surge in real-time data owing to the growing number of distributed software applications that continuously generates a large volume of data streams. These applications utilize new hardware technologies and data collection methods that inherently produce uncertain data. The Complex event processing (CEP) is a crucial software technology that manages highly dynamic event streams in a timely manner and determines the event patterns of interest. The CEP processes the massive data streams from different applications that depend on the reliability of data to produce accurate results. The business intelligence requires high-level of knowledge extracted from continuous streams. Ignoring uncertainty in event processing significantly degrades the performance and accuracy. Thus, efficient processing of uncertain data is vital to improving the efficiency and the accuracy of the CEP results for better decision making in business. This survey discusses the basics of uncertainty, explores models and query processing methods to deal with uncertainty. The primary contribution of this work is the survey of CEP regarding event query language and event query processing involved in identifying the complex event patterns over uncertain events. The challenges and the future directions for handling uncertain events in CEP are discussed in detail.

[1]  Yanlei Diao,et al.  SASE: Complex Event Processing over Streams , 2006, ArXiv.

[2]  Srinath Perera,et al.  Siddhi: a second look at complex event processing architectures , 2011, GCE '11.

[3]  Christopher Ré,et al.  Event queries on correlated probabilistic streams , 2008, SIGMOD Conference.

[4]  Chetan Gupta,et al.  High-performance complex event processing using continuous sliding views , 2013, EDBT '13.

[5]  Jennifer Widom,et al.  Working Models for Uncertain Data , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[6]  Giordano Tamburrelli,et al.  Introducing uncertainty in complex event processing: model, implementation, and validation , 2014, Computing.

[7]  T. S. Jayram,et al.  OLAP over uncertain and imprecise data , 2007, The VLDB Journal.

[8]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[9]  Johannes Gehrke,et al.  Cayuga: A General Purpose Event Monitoring System , 2007, CIDR.

[10]  Joseph S. Sventek,et al.  Unification of Publish/Subscribe Systems and Stream Databases - The Impact on Complex Event Processing , 2012, Middleware.

[11]  Philip S. Yu,et al.  A Survey of Uncertain Data Algorithms and Applications , 2009, IEEE Transactions on Knowledge and Data Engineering.

[12]  Jianhua Wang,et al.  A Complex Event Detection Method for Multi-probability RFID Event Stream , 2014, J. Softw..

[13]  Luc De Raedt,et al.  On the implementation of the probabilistic logic programming language ProbLog , 2010, Theory and Practice of Logic Programming.

[14]  Jennifer Widom,et al.  Models and issues in data stream systems , 2002, PODS.

[15]  Donald Perlis,et al.  Uniform accountability for multiple models of reasoning , 1988 .

[16]  Neil Immerman,et al.  Recognizing patterns in streams with imprecise timestamps , 2010, Proc. VLDB Endow..

[17]  Peter J. Haas,et al.  MCDB: a monte carlo approach to managing uncertain data , 2008, SIGMOD Conference.

[18]  Michael Eckert,et al.  Complex Event Processing (CEP) , 2009, Informatik-Spektrum.

[19]  Xin Li,et al.  Complex Event Processing over Uncertain Data Streams , 2010, 2010 International Conference on P2P, Parallel, Grid, Cloud and Internet Computing.

[20]  Peter R. Pietzuch,et al.  A Framework for Event Composition in Distributed Systems , 2003, Middleware.

[21]  Avigdor Gal,et al.  Complex event processing over uncertain data , 2008, DEBS.

[22]  Szabolcs Rozsnyai,et al.  SARI-SQL: Event Query Language for Event Analysis , 2009, 2009 IEEE Conference on Commerce and Enterprise Computing.

[23]  Daisy Zhe Wang,et al.  Probabilistic Complex Event Triggering , 2009 .

[24]  Peter R. Pietzuch,et al.  Distributed complex event processing with query rewriting , 2009, DEBS '09.

[25]  Amihai Motro,et al.  Uncertainty Management in Information Systems: From Needs to Solution , 1996 .

[26]  Jennifer Widom,et al.  Trio: A System for Integrated Management of Data, Accuracy, and Lineage , 2004, CIDR.

[27]  Michael Stonebraker,et al.  Aurora: a data stream management system , 2003, SIGMOD '03.

[28]  Daniele Braga,et al.  Querying RDF streams with C-SPARQL , 2010, SGMD.

[29]  Avigdor Gal,et al.  A Model for Reasoning with Uncertain Rules in Event Composition Systems , 2005, UAI.

[30]  Zhang Xiaoming,et al.  Complex Event Processing over Distributed Uncertain Event Streams , 2014 .

[31]  Hilbert J. Kappen,et al.  Bayesian Networks for Expert Systems: Theory and Practical Applications , 2010, Interactive Collaborative Information Systems.

[32]  Jennifer Widom,et al.  The CQL continuous query language: semantic foundations and query execution , 2006, The VLDB Journal.

[33]  Ambuj K. Singh,et al.  Top-k Spatial Joins of Probabilistic Objects , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[34]  Govindasamy Vaiyapuri,et al.  Probabilistic and fuzzy logic based event processing for effective business intelligence , 2016, Int. Arab J. Inf. Technol..

[35]  B. Seeger,et al.  PIPES : A Multi-Threaded Publish-Subscribe Architecture for Continuous Queries over Streaming Data Sources , 2003 .

[36]  Jennifer Widom,et al.  STREAM: The Stanford Data Stream Management System , 2016, Data Stream Management.

[37]  Ugur Çetintemel,et al.  Plan-based complex event detection across distributed sources , 2008, Proc. VLDB Endow..

[38]  Charu C. Aggarwal,et al.  Managing and Mining Uncertain Data , 2009, Advances in Database Systems.

[39]  Dan Suciu,et al.  Efficient query evaluation on probabilistic databases , 2004, The VLDB Journal.

[40]  Yongheng Wang,et al.  A Proactive Complex Event Processing Method for Large-Scale Transportation Internet of Things , 2014, Int. J. Distributed Sens. Networks.

[41]  Alex Bateman,et al.  An introduction to hidden Markov models. , 2007, Current protocols in bioinformatics.

[42]  Hiroyuki Kitagawa,et al.  Efficient probabilistic event stream processing with lineage and Kleene-plus , 2009 .

[43]  Aoying Zhou,et al.  Approximately Processing Multi-granularity Aggregate Queries over Data Streams , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[44]  Ying Xing,et al.  The Design of the Borealis Stream Processing Engine , 2005, CIDR.

[45]  Hans-Arno Jacobsen,et al.  Composite Subscriptions in Content-Based Publish/Subscribe Systems , 2005, Middleware.

[46]  Samuel Madden,et al.  ZStream: a cost-based query processor for adaptively detecting composite events , 2009, SIGMOD Conference.

[47]  Opher Etzion,et al.  Event processing under uncertainty , 2012, DEBS.

[48]  M. Balazinska,et al.  PEEX : Extracting Probabilistic Events from RFID Data , 2007 .

[49]  L. Zadeh.  The role of fuzzy logic in the management of uncertainty in expert systems , 1983 .

[50]  Lotfi A. Zadeh,et al.  Fuzzy logic = computing with words , 1996, IEEE Trans. Fuzzy Syst..

[51]  Ben Taskar,et al.  Bayesian Logic Programming: Theory and Tool , 2007 .

[52]  Yongheng Wang,et al.  Context-aware Distributed Complex Event Processing Method for Event Cloud in Internet of Things , 2013 .

[53]  Michael I. Jordan.  Graphical Models , 2003 .

[54]  Daniel L. Sherrell,et al.  Communications of the Association for Information Systems , 1999 .

[55]  Jianzhong Qiao,et al.  Complex Event Detection in Probabilistic Stream , 2010, 2010 12th International Asia-Pacific Web Conference.

[56]  Sunil Prabhakar,et al.  Evaluating probabilistic queries over imprecise data , 2003, SIGMOD '03.

[57]  Sebastian Rudolph,et al.  EP-SPARQL: a unified language for event processing and stream reasoning , 2011, WWW.

[58]  Sebastian Rudolph,et al.  ETALIS: Rule-Based Reasoning in Event Processing , 2011 .

[59]  Weiru Liu,et al.  Event Modelling and Reasoning with Uncertain Information for Distributed Sensor Networks , 2010, SUM.

[60]  Carlo Zaniolo,et al.  High-performance complex event processing over XML streams , 2012, SIGMOD Conference.

[61]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[62]  Xiaoming Zhang,et al.  Complex Event Processing over distributed probabilistic event streams , 2012, 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery.

[63]  Kyoung Soo Bok,et al.  Efficient Complex Event Processing over RFID Streams , 2012, Int. J. Distributed Sens. Networks.

[64]  Jeffrey Scott Vitter,et al.  Efficient join processing over uncertain data , 2006, CIKM '06.

[65]  Serge Abiteboul,et al.  On the representation and querying of sets of possible worlds , 1987, SIGMOD '87.

[66]  Magdalena Balazinska,et al.  Lineage for Markovian stream event queries , 2011, MobiDE '11.

[67]  Chetan Gupta,et al.  NEEL: The Nested Complex Event Language for Real-Time Event Analytics , 2010, BIRTE.

[68]  Stanley B. Zdonik,et al.  Top-k queries on uncertain data: on score distribution and typical answers , 2009, SIGMOD Conference.

[69]  Michael Eckert,et al.  A CEP Babelfish: Languages for Complex Event Processing and Querying Surveyed , 2011 .

[70]  Neil Immerman,et al.  On complexity and optimization of expensive queries in complex event processing , 2014, SIGMOD Conference.

[71]  Efraim Turban,et al.  Business Intelligence: Second European Summer School, eBISS 2012, Brussels, Belgium, July 15-21, 2012, Tutorial Lectures , 2013 .

[72]  Avigdor Gal,et al.  Efficient Processing of Uncertain Events in Rule-Based Systems , 2012, IEEE Transactions on Knowledge and Data Engineering.

[73]  Vldb Endowment,et al.  The VLDB journal : the international journal on very large data bases. , 1992 .

[74]  Christopher Ré,et al.  Efficient Top-k Query Evaluation on Probabilistic Data , 2007, 2007 IEEE 23rd International Conference on Data Engineering.

[75]  Hans-Peter Kriegel,et al.  Probabilistic Similarity Join on Uncertain Data , 2006, DASFAA.

[76]  Dan Olteanu,et al.  Fast and Simple Relational Processing of Uncertain Data , 2007, 2008 IEEE 24th International Conference on Data Engineering.

[77]  Lei Chen,et al.  Similarity Join Processing on Uncertain Data Streams , 2011, IEEE Transactions on Knowledge and Data Engineering.

[78]  Fusheng Wang,et al.  Bridging Physical and Virtual Worlds: Complex Event Processing for RFID Data Streams , 2006, EDBT.

[79]  Giordano Tamburrelli,et al.  Learning from the past: automated rule generation for complex event processing , 2014, DEBS '14.

[80]  Laks V. S. Lakshmanan,et al.  ProbView: a flexible probabilistic database system , 1997, TODS.

[81]  Michael Zink,et al.  Capturing Data Uncertainty in High-Volume Stream Processing , 2009, CIDR.

[82]  Hector Garcia-Molina,et al.  The Management of Probabilistic Data , 1992, IEEE Trans. Knowl. Data Eng..

[83]  David Luckham,et al.  The power of events - an introduction to complex event processing in distributed enterprise systems , 2002, RuleML.

[84]  Avigdor Gal,et al.  A Taxonomy and Representation of Sources of Uncertainty in Active Systems , 2006, NGITS.

[85]  Luc De Raedt,et al.  Bayesian Logic Programming: Theory and Tool , 2007 .

[86]  Mohamed A. Soliman,et al.  Top-k Query Processing in Uncertain Databases , 2007, 2007 IEEE 23rd International Conference on Data Engineering.