Queueing analysis of relational operators for continuous data streams

Currently, stream data processing is an active area of research, which includes everything from algorithms and architectures for stream processing to modelling, and analysis of various components of a stream processing system. In this paper, we present an analysis of relational operators used for stream processing using queueing theory and study behaviors of streaming data in a query processing system. Our approach enables us to compute the fundamental performance metrics of relational operators ---select, project, and join over data streams. Furthermore, this approach establishes a way to find the probability distribution functions of both the number of tuples and the waiting time of tuples in the system. Finally, we designed and implemented a number of experiments to validate the accuracy and effectiveness of our analysis.

[1]  Leonard Kleinrock,et al.  Theory, Volume 1, Queueing Systems , 1975 .

[2]  David J. DeWitt,et al.  NiagaraCQ: a scalable continuous query system for Internet databases , 2000, SIGMOD '00.

[3]  Prabhakar Raghavan,et al.  Computing on data streams , 1999, External Memory Algorithms.

[4]  Samuel Madden,et al.  Fjording the stream: an architecture for queries over streaming sensor data , 2002, Proceedings 18th International Conference on Data Engineering.

[5]  Jennifer Widom,et al.  Continuous queries over data streams , 2001, SGMD.

[6]  Noga Alon,et al.  The space complexity of approximating the frequency moments , 1996, STOC '96.

[7]  Peter M. G. Apers,et al.  Pipelining in query execution , 1990, Proceedings. PARBASE-90: International Conference on Databases, Parallel Architectures, and Their Applications.

[8]  D. Kendall.  Some Problems in the Theory of Queues , 1951 .

[9]  Jennifer Widom,et al.  Characterizing memory requirements for queries over continuous data streams , 2002, PODS '02.

[10]  Matthias Grossglauser,et al.  On the relevance of long-range dependence in network traffic , 1996, SIGCOMM '96.

[11]  Jeffrey F. Naughton,et al.  Rate-based query optimization for streaming information sources , 2002, SIGMOD '02.

[12]  Piotr Indyk,et al.  Maintaining Stream Statistics over Sliding Windows , 2002, SIAM J. Comput..

[13]  Rajeev Motwani,et al.  Sampling from a moving window over streaming data , 2002, SODA '02.

[14]  J. Mehdi.  Stochastic Models in Queueing Theory , 1991 .

[15]  Mahesh Viswanathan,et al.  An Approximate L1-Difference Algorithm for Massive Data Streams , 2002, SIAM J. Comput..

[16]  Michael Stonebraker,et al.  Monitoring Streams - A New Class of Data Management Applications , 2002, VLDB.

[17]  K. Taira.  Proof of Theorem 1.3 , 2004 .

[18]  Michael J. Franklin,et al.  Streaming Queries over Streaming Data , 2002, VLDB.