Generic windowing support for extensible stream processing systems

Stream processing applications process high volume, continuous feeds from live data sources, employ data‐in‐motion analytics to analyze these feeds, and produce near real‐time insights with low latency. One of the fundamental characteristics of such applications is the on‐the‐fly nature of the computation, which does not require access to disk resident data. Stream processing applications store the most recent history of streams in memory and use it to perform the necessary modeling and analysis tasks. This recent history is often managed using windows. All data stream management systems provide some form of windowing functionality. Windowing makes it possible to implement streaming versions of the traditionally blocking relational operators, such as streaming aggregations, joins, and sorts, as well as any other analytic operator that requires keeping the most recent tuples as state, such as time series analysis operators and signal processing operators.

[1]  JÜRGEN KRÄMER,et al.  Semantics and implementation of continuous sliding window queries over data streams , 2009, TODS.

[2]  Jennifer Widom,et al.  Resource Sharing in Continuous Sliding-Window Aggregates , 2004, VLDB.

[3]  Philip S. Yu,et al.  Challenges and Experience in Prototyping a Multi-Modal Stream Analytic and Monitoring Application on System S , 2007, VLDB.

[4]  Laura M. Haas,et al.  SECRET: A Model for Analysis of the Execution Semantics of Stream Processing Systems , 2010, Proc. VLDB Endow..

[5]  Navendu Jain,et al.  Design, implementation, and evaluation of the linear road bnchmark on the stream processing core , 2006, SIGMOD Conference.

[6]  Jennifer Widom,et al.  STREAM: The Stanford Stream Data Manager , 2003, IEEE Data Eng. Bull..

[7]  Jennifer Widom,et al.  The CQL continuous query language: semantic foundations and query execution , 2006, The VLDB Journal.

[8]  Jennifer Widom,et al.  Memory-Limited Execution of Windowed Stream Joins , 2004, VLDB.

[9]  Ling Liu,et al.  Encyclopedia of Database Systems , 2009, Encyclopedia of Database Systems.

[10]  Philip S. Yu,et al.  SPADE: the system s declarative stream processing engine , 2008, SIGMOD Conference.

[11]  Philip S. Yu,et al.  A Survey of Synopsis Construction in Data Streams , 2007, Data Streams - Models and Algorithms.

[12]  Lukasz Golab,et al.  Multi-query optimization of sliding window aggregates by schedule synchronization , 2006, CIKM '06.

[13]  Frederick Reiss,et al.  TelegraphCQ: Continuous Dataflow Processing for an Uncertain World , 2003, CIDR.

[14]  Philip S. Yu,et al.  Processing high data rate streams in System S , 2011, J. Parallel Distributed Comput..

[15]  David Maier,et al.  Exploiting Punctuation Semantics in Continuous Data Streams , 2003, IEEE Trans. Knowl. Data Eng..

[16]  SeegerBernhard,et al.  Semantics and implementation of continuous sliding window queries over data streams , 2009 .

[17]  Philip S. Yu,et al.  GrubJoin: An Adaptive, Multi-Way, Windowed Stream Join with Time Correlation-Aware CPU Load Shedding , 2007, IEEE Transactions on Knowledge and Data Engineering.

[18]  Deepak S. Turaga,et al.  Design principles for developing stream processing applications , 2010, Softw. Pract. Exp..

[19]  Ying Xing,et al.  The Design of the Borealis Stream Processing Engine , 2005, CIDR.

[20]  David Maier,et al.  Semantics and evaluation techniques for window aggregates in data streams , 2005, SIGMOD '05.

[21]  Bugra Gedik,et al.  A model‐based framework for building extensible, high performance stream processing middleware and programming language for IBM InfoSphere Streams , 2012, Softw. Pract. Exp..

[22]  Badrish Chandramouli,et al.  Spatio-Temporal Stream Processing in Microsoft StreamInsight , 2010, IEEE Data Eng. Bull..

[23]  Laura M. Haas,et al.  Modeling the execution semantics of stream processing engines with SECRET , 2012, The VLDB Journal.

[24]  Theodore Johnson,et al.  Gigascope: a stream database for network applications , 2003, SIGMOD '03.