Real-time scheduling for data stream management systems

Quality-aware management of data streams is gaining more and more importance with the amount of data produced by streams growing continuously. The resources required for data stream processing depend on different factors and are limited by the environment of the data stream management system (DSMS). Thus, with a potentially unbounded amount of stream data and limited processing resources, some of the data stream processing tasks (originating from different users) may not be satisfyingly answered, and therefore, users should be enabled to negotiate a certain quality for the execution of their stream processing tasks. After the negotiation process, it is the responsibility of the Data Stream Management System to meet the quality constraints by using adequate resource reservation and scheduling techniques. Within this paper, we consider different aspects of real-time scheduling for operations within a DSMS. We propose a scheduling concept which enables us to meet certain time-dependent quality of service requirements for user-given processing tasks. Furthermore, we describe the implementation of our scheduling concept within a real-time capable data stream management system, and we give experimental results on that.

[1]  Hermann Härtig,et al.  DROPS: OS support for distributed multimedia applications , 1998, ACM SIGOPS European Workshop.

[2]  Frederick Reiss,et al.  TelegraphCQ: Continuous Dataflow Processing for an Uncertain World , 2003, CIDR.

[3]  Dan Hildebrand,et al.  An Architectural Overview of QNX , 1992, USENIX Workshop on Microkernels and Other Kernel Architectures.

[4]  Bernhard Seeger,et al.  PIPES: a public infrastructure for processing and exploring streams , 2004, SIGMOD '04.

[5]  Wolfgang Lehner,et al.  QStream: Deterministic Querying of Data Streams , 2004, VLDB.

[6]  Wolfgang Lehner,et al.  Integrated resource management for data stream systems , 2005, SAC '05.

[7]  Theodore Johnson,et al.  Gigascope: a stream database for network applications , 2003, SIGMOD '03.

[8]  Paolo Mantegazza,et al.  Real time distributed control systems using RTAI , 2003, Sixth IEEE International Symposium on Object-Oriented Real-Time Distributed Computing, 2003..

[9]  Anne Rogers,et al.  Hancock: a language for extracting signatures from data streams , 2000, KDD '00.

[10]  Jane W.-S. Liu.  Real-Time Systems , 2000, Encyclopedia of Algorithms.

[11]  Victor Yodaiken,et al.  A Real-Time Linux , 2000 .

[12]  Claude-Joachim Hamann,et al.  On the quantitative specification of jitter constrained periodic streams , 1997, Proceedings Fifth International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems.

[13]  Michael Stonebraker,et al.  Monitoring Streams - A New Class of Data Management Applications , 2002, VLDB.

[14]  Rajeev Motwani,et al.  Chain: operator scheduling for memory minimization in data stream systems , 2003, SIGMOD '03.