PAC: Pattern-driven Application Consolidation for Efficient Cloud Computing

To reduce cloud system resource cost, application consolidation is a must. In this paper, we present a novel pattern driven application consolidation (PAC) system to achieve efficient resource sharing in virtualized cloud computing infrastructures. PAC employs signal processing techniques to dynamically discover significant patterns called signatures of different applications and hosts. PAC then performs dynamic application consolidation based on the extracted signatures. We have implemented a prototype of the PAC system on top of the Xen virtual machine platform and tested it on the NCSU Virtual Computing Lab. We have tested our system using RUBiS benchmarks, Hadoop data processing systems, and IBM System S stream processing system. Our experiments show that 1) PAC can efficiently discover repeating resource usage patterns in the tested applications; 2) Signatures can reduce resource prediction errors by 50-90% compared to traditional coarse-grained schemes; 3) PAC can improve application performance by up to 50% when running a large number of applications on a shared cluster.

[1]  Armando Fox,et al.  Capturing, indexing, clustering, and retrieving system history , 2005, SOSP '05.

[2]  Ludmila Cherkasova,et al.  Supporting Application QoS in Shared Resource Pools , 2006 .

[3]  Dimitrios Gunopulos,et al.  Indexing multi-dimensional time-series with support for multiple distance measures , 2003, KDD '03.

[4]  Jerome A. Rolia,et al.  An integrated approach to resource pool management: Policies, efficiency and quality metrics , 2008, 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN).

[5]  Amin Vahdat,et al.  Enforcing Performance Isolation Across Virtual Machines in Xen , 2006, Middleware.

[6]  David Sankoff,et al.  Time Warps, String Edits, and Macromolecules: The Theory and Practice of Sequence Comparison , 1983 .

[7]  Gargi Dasgupta,et al.  Server Workload Analysis for Power Minimization using Consolidation , 2009, USENIX Annual Technical Conference.

[8]  Anand Sivasubramaniam,et al.  Xen and co.: communication-aware CPU scheduling for consolidated xen-based hosting platforms , 2007, VEE '07.

[9]  Shane S. Sturrock,et al.  Time Warps, String Edits, and Macromolecules – The Theory and Practice of Sequence Comparison . David Sankoff and Joseph Kruskal. ISBN 1-57586-217-4. Price £13.95 (US$22·95). , 2000 .

[10]  Ming Zhong,et al.  I/O system performance debugging using model-driven anomaly characterization , 2005, FAST'05.

[11]  Andrzej Kochut.  On Impact of Dynamic Virtual Machine Reallocation on Data Center Efficiency , 2008, 2008 IEEE International Symposium on Modeling, Analysis and Simulation of Computers and Telecommunication Systems.

[12]  Prashant J. Shenoy,et al.  Dynamic resource allocation for shared data centers using online measurements , 2003, IWQoS'03.

[13]  Jerome A. Rolia,et al.  Capacity Management and Demand Prediction for Next Generation Data Centers , 2007, IEEE International Conference on Web Services (ICWS 2007).

[14]  Alan L. Cox,et al.  Optimizing network virtualization in Xen , 2006 .

[15]  Suman Nath,et al.  Energy-Aware Server Provisioning and Load Dispatching for Connection-Intensive Internet Services , 2008, NSDI.

[16]  Andrew Warfield,et al.  Live migration of virtual machines , 2005, NSDI.

[17]  David E. Culler,et al.  A blueprint for introducing disruptive technology into the Internet , 2003, CCRV.

[18]  Philip S. Yu,et al.  SPADE: the system s declarative stream processing engine , 2008, SIGMOD Conference.

[19]  Stan Salvador,et al.  FastDTW: Toward Accurate Dynamic Time Warping in Linear Time and Space , 2004 .

[20]  Xiaoyun Zhu,et al.  1000 Islands: Integrated Capacity and Workload Management for the Next Generation Data Center , 2008, 2008 International Conference on Autonomic Computing.

[21]  Anand Sivasubramaniam,et al.  Profiling, Prediction, and Capping of Power Consumption in Consolidated Environments , 2008, 2008 IEEE International Symposium on Modeling, Analysis and Simulation of Computers and Telecommunication Systems.

[22]  Philip S. Yu,et al.  Challenges and Experience in Prototyping a Multi-Modal Stream Analytic and Monitoring Application on System S , 2007, VLDB.

[23]  Qi Zhang,et al.  Autocorrelation-driven load control in distributed systems , 2009, 2009 IEEE International Symposium on Modeling, Analysis & Simulation of Computer and Telecommunication Systems.

[24]  Klara Nahrstedt,et al.  Dynamic QoS-aware multimedia service configuration in ubiquitous computing environments , 2002, Proceedings 22nd International Conference on Distributed Computing Systems.

[25]  Andrzej Kochut,et al.  Dynamic Placement of Virtual Machines for Managing SLA Violations , 2007, 2007 10th IFIP/IEEE International Symposium on Integrated Network Management.

[26]  Xiaosong Ma,et al.  SigLM: Signature-driven load management for cloud computing infrastructures , 2009, 2009 17th International Workshop on Quality of Service.

[27]  Jerome A. Rolia,et al.  Supporting application quality of service in shared resource pools , 2006, Commun. ACM.

[28]  Ludmila Cherkasova,et al.  An Automated Approach for Supporting Application QoS in Shared Resource Pools , 2005 .

[29]  Jerome A. Rolia,et al.  Satisfying Service Level Objectices in a Self-Managing Resource Pool , 2009, 2009 Third IEEE International Conference on Self-Adaptive and Self-Organizing Systems.

[30]  Dhabaleswar K. Panda,et al.  High Performance VMM-Bypass I/O in Virtual Machines , 2006, USENIX Annual Technical Conference, General Track.

[31]  Joseph B. Kruskall,et al.  The Symmetric Time-Warping Problem : From Continuous to Discrete , 1983 .

[32]  Carl A. Waldspurger,et al.  Memory resource management in VMware ESX server , 2002, OSDI '02.

[33]  Michele Colajanni,et al.  Runtime Prediction Models for Internet-based Systems , 2008, MASCOTS.

[34]  Archana Ganapathi,et al.  Predicting Multiple Metrics for Queries: Better Decisions Enabled by Machine Learning , 2009, 2009 IEEE 25th International Conference on Data Engineering.