Load shedding techniques based on windows in data stream systems

Many applications need to process streams, for analyzing and monitoring their data for inferring useful information. Database analysts are structuring Data Stream Management systems (DSMS) so that applications can subject queries to get timely information from streams. Unlike the traditional database system, managing and processing of stream database raise several challenges. In this paper, we portrayed a new load shedding system for queries consisting of one or more aggregate operators with sliding windows. We utilized different types of window aggregate function to drop the tuple in DataStream. This method is conscious of the window properties of its window aggregate operators in the query plan. Accordingly, it plausibly divides the input stream into windows and probabilistically decides which tuple to drop based on the window function. This decision is further encoded into tuple by marking the ones that are disallowed from starting new windows. Unlike previous methods, our method conserve consistency of windows all over a query plan, and always distributes subsets of original query responds with negligible deprivation in the quality of the result.

[1]  Shonali Krishnaswamy,et al.  Mining data streams: a review , 2005, SGMD.

[2]  Doaa S. El Zanfaly,et al.  Prioritized query shedding technique for continuous queries over data streams , 2009, 2009 International Conference on Computer Engineering & Systems.

[3]  Georges Hébrail,et al.  Data stream management and mining , 2007, NATO ASI Mining Massive Data Sets for Security.

[4]  Liu Quan,et al.  Study of Agent Model by Combining Logic and Economics Approach , 2007, 2007 International Conference on Convergence Information Technology (ICCIT 2007).

[5]  Stefanos Manganaris,et al.  A Data Mining Analysis of RTID Alarms , 2000, Recent Advances in Intrusion Detection.

[6]  Kuen-Fang Jea,et al.  A load shedding scheme for frequent pattern mining in transactional data streams , 2011, 2011 Eighth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD).

[7]  Rajeev Motwani,et al.  Load shedding for aggregation queries over data streams , 2004, Proceedings. 20th International Conference on Data Engineering.

[8]  Mohamed Medhat Gaber,et al.  Data Stream Mining , 2010, Data Mining and Knowledge Discovery Handbook.

[9]  He Chun-liang.  Load shedding for sliding window aggregation queries over data streams , 2009 .

[10]  Li Ma,et al.  Semantic Load Shedding over Real-Time Data Streams , 2008, 2008 International Symposium on Computational Intelligence and Design.

[11]  Yu Min,et al.  Semantic Load Shedding for Sliding Window Join-Aggregation Queries over Data Streams , 2007, 2007 International Conference on Convergence Information Technology (ICCIT 2007).

[12]  WeiWei.  A Novel Adaptive Load Shedding Scheme for Data Stream Processing , 2008 .

[13]  Li Ma,et al.  A semantic load shedding algorithm based on priority table in Data Stream System , 2010, 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery.

[14]  Kuen-Fang Jea,et al.  A load-controllable mining system for frequent-pattern discovery in dynamic data streams , 2010, 2010 International Conference on Machine Learning and Cybernetics.