Automated and dynamic application accuracy management and resource provisioning in a cloud environment

The recent emergence of cloud computing is making the vision of utility computing realizable, i.e., computing resources and services from a cloud can be delivered, utilized, and paid for in the same fashion as utilities like water or electricity. This, however, creates new resource provisioning problems. Because of the pay-as-you-go model, resource provisioning should be performed carefully. Resource provisioning can be particularly challenging for adaptive applications, where there can be a tradeoff between the application Quality of Service (QoS), or accuracy, and the resource costs incurred. In this paper, we consider adaptive streaming applications where a user wants to achieve the minimum resource costs while maintaining a specified accuracy goal. We present a dynamic and automated framework which can adapt the adaptive parameters to meet the specific accuracy goal, and then dynamically converge to near-optimal resource allocation. Our solution can handle unexpected changes in the data distribution characteristics and/or rates. We evaluate our approach using two streaming applications and demonstrate the effectiveness of our framework.

[1]  Rajeev Motwani,et al.  Approximate Frequency Counts over Data Streams , 2012, VLDB.

[2]  Liang Chen,et al.  GATES: a grid-based middleware for processing distributed data streams , 2004, Proceedings. 13th IEEE International Symposium on High performance Distributed Computing, 2004..

[3]  Philip S. Yu,et al.  A Framework for Clustering Evolving Data Streams , 2003, VLDB.

[4]  Rajeev Motwani,et al.  Operator scheduling in data stream systems , 2004, VLDB 2004.

[5]  Douglas F. Parkhill,et al.  The Challenge of the Computer Utility , 1966 .

[6]  Steven Hand,et al.  Self-adaptive and self-configured CPU resource provisioning for virtualized servers using Kalman filters , 2009, ICAC '09.

[7]  Karsten Schwan,et al.  Dynamic Querying of Streaming Data with the dQUOB System , 2003, IEEE Trans. Parallel Distributed Syst..

[8]  Kun-Lung Wu,et al.  Elastic scaling of data parallel operators in stream processing , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[9]  Andrew Warfield,et al.  Xen and the art of virtualization , 2003, SOSP '03.

[10]  Michael Stonebraker,et al.  Fault-tolerance in the borealis distributed stream processing system , 2008, ACM Trans. Database Syst..

[11]  Sang-Min Park,et al.  Feedback-controlled resource sharing for predictable eScience , 2008, HiPC 2008.

[12]  Michael Stonebraker,et al.  The Aurora and Medusa Projects , 2003, IEEE Data Eng. Bull..

[13]  Navendu Jain,et al.  Design, implementation, and evaluation of the linear road bnchmark on the stream processing core , 2006, SIGMOD Conference.

[14]  Mohamed Medhat Gaber,et al.  Data Stream Mining Using Granularity-Based Approach , 2009, Foundations of Computational Intelligence.

[15]  Kang G. Shin,et al.  Adaptive control of virtualized resources in utility computing environments , 2007, EuroSys '07.

[16]  Anand Sivasubramaniam,et al.  Xen and co.: communication-aware CPU scheduling for consolidated xen-based hosting platforms , 2007, VEE '07.