Tuning complex event processing rules using the prediction-correction paradigm

There is a growing need for the use of active systems, systems that act automatically based on events. In many cases, providing such active functionality requires materializing (inferring) the occurrence of relevant events. A widespread paradigm for enabling such materialization is Complex Event Processing (CEP), a rule based paradigm, which currently relies on domain experts to fully define the relevant rules. These experts need to provide the set of basic events which serves as input to the rule, their inter-relationships, and the parameters of the events for determining a new event materialization. While it is reasonable to expect that domain experts will be able to provide a partial rules specification, providing all the required details is a hard task, even for domain experts. Moreover, in many active systems, rules may change over time, due to the dynamic nature of the domain. Such changes complicate even further the specification task, as the expert must constantly update the rules. As a result, we seek additional support to the definition of rules, beyond expert opinion. This work presents a mechanism for automating both the initial definition of rules and the update of rules over time. This mechanism combines partial information provided by the domain expert with machine learning techniques, and is aimed at improving the accuracy of event specification and materialization. The proposed mechanism consists of two main repetitive stages, namely rule parameter prediction and rule parameter correction. The former is performed by updating the parameters using an available expert knowledge regarding the future changes of parameters. The latter stage utilizes expert feedback regarding the actual past occurrence of events and the events materialized by the CEP framework to tune rule parameters. We also include possible implementations for both stages, based on a statistical estimator and evaluate our outcome using a case study from the intrusion detection domain.

[1]  Sharma Chakravarthy,et al.  Snoop: An Expressive Event Specification Language for Active Databases , 1994, Data Knowl. Eng..

[2]  Fred W. Glover,et al.  Tabu Search , 1997, Handbook of Heuristics.

[3]  Kurt Rothermel,et al.  An architecture for observing physical world events , 2005, 11th International Conference on Parallel and Distributed Systems (ICPADS'05).

[4]  Opher Etzion,et al.  Amit - the situation manager , 2003, The VLDB Journal.

[5]  Dan Suciu,et al.  Probabilistic Event Extraction from RFID Data , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[6]  Wei Hong,et al.  Approximate Data Collection in Sensor Networks using Probabilistic Models , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[7]  Johannes Gehrke,et al.  Cayuga: A General Purpose Event Monitoring System , 2007, CIDR.

[8]  Avigdor Gal,et al.  Generic Architecture of Complex Event Processing Systems , 2010, Principles and Applications of Distributed Event-Based Systems.

[9]  M. Balazinska,et al.  PEEX : Extracting Probabilistic Events from RFID Data , 2007 .

[10]  A. N. Zincir-Heywood,et al.  Intrusion Detection Systems , 2008 .

[11]  Manas Ranjan Patra,et al.  NETWORK INTRUSION DETECTION USING NAÏVE BAYES , 2007 .

[12]  S. Griffis.  EDITOR , 1997, Journal of Navigation.

[13]  Avigdor Gal,et al.  Complex event processing over uncertain data , 2008, DEBS.

[14]  Amit P. Sheth,et al.  A Semantic Framework for Identifying Events in a Service Oriented Architecture , 2007, IEEE International Conference on Web Services (ICWS 2007).

[15]  David Luckham,et al.  The power of events - an introduction to complex event processing in distributed enterprise systems , 2002, RuleML.

[16]  Arthur Gelb,et al.  Applied Optimal Estimation , 1974 .

[17]  Rainer von Ammon.  Event-Driven Business Process Management , 2009, Encyclopedia of Database Systems.

[18]  Christof Bornhövd,et al.  Event Handling for the Universal Enterprise , 2005, Inf. Technol. Manag..

[19]  Lionel Sacks,et al.  Active Platform Security through Intrusion Detection Using Naïve Bayesian Network for Anomaly Detection , 2002 .

[20]  Dennis Heimbigner,et al.  Event-based Document Sensing for Insider Threats , 2004 .

[21]  Alfonso Fuggetta,et al.  The JEDI Event-Based Infrastructure and Its Application to the Development of the OPSS WFMS , 2001, IEEE Trans. Software Eng..

[22]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[23]  Peter R. Pietzuch,et al.  Composite event detection as a generic middleware extension , 2004, IEEE Network.

[24]  Hartmut Ritter,et al.  Fence Monitoring - Experimental Evaluation of a Use Case for Wireless Sensor Networks , 2007, EWSN.

[25]  Jun Wang,et al.  A metamodel for distributed event based systems , 2008, DEBS.

[26]  Ying Li,et al.  JTang Synergy: A Service Oriented Architecture for Enterprise Application Integration , 2007, 2007 11th International Conference on Computer Supported Cooperative Work in Design.

[27]  Ricardo Staciarini Puttini,et al.  A Bayesian Classification Model for Real‐Time Intrusion Detection , 2003 .

[28]  Christopher Krügel,et al.  Bayesian event classification for intrusion detection , 2003, 19th Annual Computer Security Applications Conference, 2003. Proceedings..

[29]  Stefan Axelsson,et al.  Intrusion Detection Systems: A Survey and Taxonomy , 2002 .

[30]  Charu C. Aggarwal,et al.  AN EVALUATION OF OVER-THE-COUNTER MEDICATION SALES FOR SYNDROMIC SURVEILLANCE , 2006 .

[31]  Seth White,et al.  WebLogic event server: a lightweight, modular application server for event processing , 2008, DEBS.

[32]  Alfonso Valdes,et al.  Adaptive, Model-Based Monitoring for Cyber Attack Detection , 2000, Recent Advances in Intrusion Detection.

[33]  Norman W. Paton,et al.  DEAR: a DEbugger for Active Rules in an object-oriented context , 1993, Rules in Database Systems.

[34]  Shie Mannor,et al.  A Tutorial on the Cross-Entropy Method , 2005, Ann. Oper. Res..

[35]  Avigdor Gal,et al.  Inference of Security Hazards from Event Composition Based on Incomplete or Uncertain Information , 2008, IEEE Transactions on Knowledge and Data Engineering.

[36]  John McHugh,et al.  Intrusion and intrusion detection , 2001, International Journal of Information Security.

[37]  Beth Plale,et al.  Prediction of Missing Events in Sensor Data Streams Using Kalman Filters , 2007 .

[38]  Peter Mell,et al.  Intrusion Detection Systems , 2001 .

[39]  Andrzej Cichocki,et al.  Event-driven Video Awareness Providing Physical Security , 2007, World Wide Web.

[40]  Charles Elkan,et al.  Results of the KDD'99 classifier learning , 2000, SKDD.