Complex Event Processing over distributed probabilistic event streams

With the rapid development of Internet of Things (IoT), enormous events are produced everyday. Complex Event Processing (CEP) is the key part of the IoT middleware. Since current hardware and wireless communication techniques cannot support 100% confident data, CEP engine which can report confidence for processed complex events over uncertain data is needed. Most of the current study of complex event processing has not considered much about how to process complex event over distributed probabilistic event streams and large sliding window. In this paper, a high performance complex event processing method over distributed probabilistic event streams is proposed. This method uses probabilistic Nondeterministic Finite Automaton and Active Instance Stacks to process complex event in single probabilistic event stream. Multiple processes can run parallel to improve the performance. A query plan based method using tree data structure is used to process hierarchical complex event from distributed event streams. Query plan optimization is proposed based on query optimization technology of probabilistic databases. The experimental study shows that this method is efficient to process complex events over distributed probabilistic event streams.

[1]  Xin Li,et al.  Complex Event Processing over Uncertain Data Streams , 2010, 2010 International Conference on P2P, Parallel, Grid, Cloud and Internet Computing.

[2]  Henry A. Kautz,et al.  Location-Based Reasoning about Complex Multi-Agent Behavior , 2012, J. Artif. Intell. Res..

[3]  Yanlei Diao,et al.  High-performance complex event processing over streams , 2006, SIGMOD Conference.

[4]  Klaus R. Dittrich,et al.  Events in an Active Object-Oriented Database System , 1993, Rules in Database Systems.

[5]  Hendrik Segers,et al.  Composite event specification in active databases: model and implementation , 1992 .

[6]  Luc De Raedt,et al.  On the implementation of the probabilistic logic programming language ProbLog , 2010, Theory and Practice of Logic Programming.

[7]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[8]  Opher Etzion,et al.  Event processing under uncertainty , 2012, DEBS.

[9]  David Luckham,et al.  The power of events - an introduction to complex event processing in distributed enterprise systems , 2002, RuleML.

[10]  Patrick Weber,et al.  OpenStreetMap: User-Generated Street Maps , 2008, IEEE Pervasive Computing.

[11]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[12]  Neil Immerman,et al.  Recognizing patterns in streams with imprecise timestamps , 2010, Proc. VLDB Endow..

[13]  Neil Immerman,et al.  Efficient pattern matching over event streams , 2008, SIGMOD Conference.

[14]  Hiroyuki Kitagawa,et al.  Probabilistic Event Stream Processing with Lineage , 2008 .

[15]  Dan Suciu,et al.  Efficient query evaluation on probabilistic databases , 2004, The VLDB Journal.

[16]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[17]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[18]  Opher Etzion,et al.  Event Processing in Action , 2010 .

[19]  Yushun Fan,et al.  Complex event processing in enterprise information systems based on RFID , 2007, Enterp. Inf. Syst..

[20]  Sebastian Thrun,et al.  Recognizing Activities with Multiple Cues , 2007, Workshop on Human Motion.

[21]  Sharma Chakravarthy,et al.  Composite Events for Active Databases: Semantics, Contexts and Detection , 1994, VLDB.

[22]  Opher Etzion,et al.  Event-processing network model and implementation , 2008, IBM Syst. J..

[23]  Larry S. Davis,et al.  Multi-agent event recognition in structured scenarios , 2011, CVPR 2011.

[24]  Randal E. Bryant,et al.  Graph-Based Algorithms for Boolean Function Manipulation , 1986, IEEE Transactions on Computers.

[25]  Dan Suciu,et al.  Management of probabilistic data: foundations and challenges , 2007, PODS '07.

[26]  Zhitao Shen,et al.  Lineage-based Probabilistic Event Stream Processing , 2008, 2008 Ninth International Conference on Mobile Data Management Workshops, MDMW.

[27]  Jianzhong Qiao,et al.  Complex Event Detection in Probabilistic Stream , 2010, 2010 12th International Asia-Pacific Web Conference.

[28]  Charu C. Aggarwal,et al.  Trio A System for Data Uncertainty and Lineage , 2009 .

[29]  Jun Wang,et al.  Accelerating Sequence Event Detection through Condensed Composition , 2010, 2010 Proceedings of the 5th International Conference on Ubiquitous Information Technologies and Applications.

[30]  Annika Hinze.  Efficient Filtering of Composite Events , 2003, BNCOD.

[31]  Lin Nan,et al.  A Novel Distributed Complex Event Processing for RFID Application , 2008, 2008 Third International Conference on Convergence and Hybrid Information Technology.

[32]  Ugur Çetintemel,et al.  Plan-based complex event detection across distributed sources , 2008, Proc. VLDB Endow..

[33]  Yongheng Wang,et al.  Plan Based Parallel Complex Event Detection over RFID Streams , 2009, 2009 First International Conference on Information Science and Engineering.

[34]  Fusheng Wang,et al.  Bridging Physical and Virtual Worlds: Complex Event Processing for RFID Data Streams , 2006, EDBT.