A Predictive-Reactive Method for Improving the Robustness of Real-Time Data Services

Supporting timely data services using fresh data in data-intensive real-time applications, such as e-commerce and transportation management is desirable but challenging, since the workload may vary dynamically. To control the data service delay to be below the specified threshold, we develop a predictive as well as reactive method for database admission control. The predictive method derives the workload bound for admission control in a predictive manner, making no statistical or queuing-theoretic assumptions about workloads. Also, our reactive scheme based on formal feedback control theory continuously adjusts the database load bound to support the delay threshold. By adapting the load bound in a proactive fashion, we attempt to avoid severe overload conditions and excessive delays before they occur. Also, the feedback control scheme enhances the timeliness by compensating for potential prediction errors due to dynamic workloads. Hence, the predictive and reactive methods complement each other, enhancing the robustness of real-time data services as a whole. We implement the integrated approach and several baselines in an open-source database. Compared to the tested open-loop, feedback-only, and statistical prediction + feedback baselines representing the state of the art, our integrated method significantly improves the average/transient delay and real-time data service throughput.

[1]  Sang Hyuk Son,et al.  Chronos: Feedback Control of a Real Database System Performance , 2007, 28th IEEE International Real-Time Systems Symposium (RTSS 2007).

[2]  Lothar Thiele,et al.  Composing Functional and State-Based Performance Models for Analyzing Heterogeneous Real-Time Systems , 2007, RTSS 2007.

[3]  Badrish Chandramouli,et al.  Accurate latency estimation in a distributed event processing system , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[4]  Sang Hyuk Son,et al.  Feedback Control Real-Time Scheduling: Framework, Modeling, and Algorithms* , 2001, Real-Time Systems.

[5]  B. Anderson,et al.  Digital control of dynamic systems , 1981, IEEE Transactions on Acoustics, Speech, and Signal Processing.

[6]  Richard F. Gunst,et al.  Applied Regression Analysis , 1999, Technometrics.

[7]  Raghu Ramakrishnan,et al.  Database Management Systems , 1976 .

[8]  Sang Hyuk Son,et al.  Real-Time Database Systems , 1997 .

[9]  Yixin Diao,et al.  Feedback Control of Computing Systems , 2004 .

[10]  Jörgen Hansson,et al.  Specification and management of QoS in real-time databases supporting imprecise computations , 2006, IEEE Transactions on Computers.

[11]  Allan Kuchinsky,et al.  Integrating user-perceived quality into Web server design , 2000, Comput. Networks.

[12]  Rajeev Motwani,et al.  Load shedding for aggregation queries over data streams , 2004, Proceedings. 20th International Conference on Data Engineering.

[13]  Sang Hyuk Son,et al.  Real-Time Databases and Data Services , 2004, Real-Time Systems.

[14]  Stanley B. Zdonik,et al.  Staying FIT: Efficient Load Shedding Techniques for Distributed Stream Processing , 2007, VLDB.

[15]  Michael Stonebraker,et al.  Load Shedding in a Data Stream Manager , 2003, VLDB.

[16]  Adam Wierman,et al.  How to Determine a Good Multi-Programming Level for External Scheduling , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[17]  Lui Sha,et al.  Feedback control with queueing-theoretic prediction for relative delay guarantees in web servers , 2003, The 9th IEEE Real-Time and Embedded Technology and Applications Symposium, 2003. Proceedings..

[18]  Lui Sha,et al.  Queueing model based network server performance control , 2002, 23rd IEEE Real-Time Systems Symposium, 2002. RTSS 2002..

[19]  Sang Hyuk Son,et al.  Design, Implementation, and Evaluation of a QoS-Aware Real-Time Embedded Database , 2012, IEEE Transactions on Computers.

[20]  Sang Hyuk Son,et al.  Managing deadline miss ratio and sensor data freshness in real-time databases , 2004, IEEE Transactions on Knowledge and Data Engineering.

[21]  Nesime Tatbul,et al.  Load Shedding , 2009, Encyclopedia of Database Systems.

[22]  Yan Zhou,et al.  Integrating Proactive and Reactive Approaches for Robust Real-Time Data Services , 2009, 2009 30th IEEE Real-Time Systems Symposium.

[23]  Ying Xing,et al.  The Design of the Borealis Stream Processing Engine , 2005, CIDR.

[24]  Joseph L. Hellerstein,et al.  Using Control Theory to Achieve Service Level Objectives In Performance Management , 2002, Real-Time Systems.

[25]  Sang Hyuk Son,et al.  Feedback Control Architecture and Design Methodology for Service Delay Guarantees in Web Servers , 2006, IEEE Transactions on Parallel and Distributed Systems.

[26]  Jan Lindstr̈om,et al.  Real Time Database Systems , 2008, Wiley Encyclopedia of Computer Science and Engineering.

[27]  T. Abdelzaher,et al.  Improved prediction for Web server delay control , 2004, Proceedings. 16th Euromicro Conference on Real-Time Systems, 2004. ECRTS 2004..

[28]  Eric R. Ziegel,et al.  Probability and Statistics for Engineering and the Sciences , 2004, Technometrics.

[29]  Jörgen Hansson,et al.  Generalized performance management of multi-class real-time imprecise data services , 2005, 26th IEEE International Real-Time Systems Symposium (RTSS'05).

[30]  Tarek F. Abdelzaher,et al.  Differentiated caching services; a control-theoretical approach , 2001, Proceedings 21st International Conference on Distributed Computing Systems.

[31]  Yan Zhou,et al.  Backlog Estimation and Management for Real-Time Data Services , 2008, 2008 Euromicro Conference on Real-Time Systems.