Autonomic Placement of Mixed Batch and Transactional Workloads

To reduce the cost of infrastructure and electrical energy, enterprise datacenters consolidate workloads on the same physical hardware. Often, these workloads comprise both transactional and long-running analytic computations. Such consolidation brings new performance management challenges due to the intrinsically different nature of a heterogeneous set of mixed workloads, ranging from scientific simulations to multitier transactional applications. The fact that such different workloads have different natures imposes the need for new scheduling mechanisms to manage collocated heterogeneous sets of applications, such as running a web application and a batch job on the same physical server, with differentiated performance goals. In this paper, we present a technique that enables existing middleware to fairly manage mixed workloads: long running jobs and transactional applications. Our technique permits collocation of the workload types on the same physical hardware, and leverages virtualization control mechanisms to perform online system reconfiguration. In our experiments, including simulations as well as a prototype system built on top of state-of-the-art commercial middleware, we demonstrate that our technique maximizes mixed workload performance while providing service differentiation based on high-level performance goals.

[1]  Hoon Choi,et al.  Virtual machine migration in self-managing virtualized server environments , 2009, 2009 11th International Conference on Advanced Communication Technology.

[2]  David E. Culler,et al.  User-Centric Performance Analysis of Market-Based Cluster Batch Schedulers , 2002, 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'02).

[3]  Insup Lee,et al.  On the feasibility of dynamic rescheduling on the Intel Distributed Computing Platform , 2010, Middleware Industrial Track '10.

[4]  Ulas C. Kozat,et al.  Dynamic resource allocation and power management in virtualized data centers , 2010, 2010 IEEE Network Operations and Management Symposium - NOMS 2010.

[5]  Evgenia Smirni,et al.  Efficient resource allocation and power saving in multi-tiered systems , 2010, WWW '10.

[6]  Asser N. Tantawi,et al.  Dynamic Application Placement Under Service and Memory Constraints , 2005, WEA.

[7]  Jordi Torres,et al.  Utility-based placement of dynamic Web applications with fairness goals , 2008, NOMS 2008 - 2008 IEEE Network Operations and Management Symposium.

[8]  Asser N. Tantawi,et al.  Performance management for cluster-based web services , 2005, IEEE Journal on Selected Areas in Communications.

[9]  Asser N. Tantawi,et al.  Dynamic estimation of CPU demand of web traffic , 2006, valuetools '06.

[10]  Malgorzata Steinder,et al.  A scalable application placement controller for enterprise data centers , 2007, WWW '07.

[11]  Jeffrey O. Kephart,et al.  Multi-aspect hardware management in enterprise server consolidation , 2010, 2010 IEEE Network Operations and Management Symposium - NOMS 2010.

[12]  Jing Xu,et al.  On the Use of Fuzzy Modeling in Virtualized Data Center Management , 2007, Fourth International Conference on Autonomic Computing (ICAC'07).

[13]  Jeffrey S. Chase,et al.  Automated control in cloud computing: challenges and opportunities , 2009, ACDC '09.

[14]  Xiaoyun Zhu,et al.  Capacity and Performance Overhead in Dynamic Resource Allocation to Virtual Containers , 2007, 2007 10th IFIP/IEEE International Symposium on Integrated Network Management.

[15]  Shicong Meng,et al.  Tide: achieving self-scaling in virtualized datacenter management middleware , 2010, Middleware Industrial Track '10.

[16]  Asser N. Tantawi,et al.  Dynamic placement for clustered web applications , 2006, WWW '06.

[17]  Yuan Chen,et al.  Integrated management of application performance, power and cooling in data centers , 2010, 2010 IEEE Network Operations and Management Symposium - NOMS 2010.

[18]  Aameek Singh,et al.  Shares and utilities based power consolidation in virtualized server environments , 2009, 2009 IFIP/IEEE International Symposium on Integrated Network Management.

[19]  Nicholas Bambos,et al.  Adaptive data-aware utility-based scheduling in resource-constrained systems , 2010, J. Parallel Distributed Comput..

[20]  Malgorzata Steinder,et al.  Server virtualization in autonomic management of heterogeneous workloads , 2007, 2007 10th IFIP/IEEE International Symposium on Integrated Network Management.

[21]  Gang Wang,et al.  Appliance-Based Autonomic Provisioning Framework for Virtualized Outsourcing Data Center , 2007, Fourth International Conference on Autonomic Computing (ICAC'07).

[22]  David R. Kaeli,et al.  Quantifying load imbalance on virtualized enterprise servers , 2010, WOSP/SIPEW '10.

[23]  Prashant J. Shenoy,et al.  Resource overbooking and application profiling in shared hosting platforms , 2002, OSDI '02.

[24]  Michael I. Jordan,et al.  Statistical Machine Learning Makes Automatic Control Practical for Internet Datacenters , 2009, HotCloud.

[25]  Cynthia Bailey Lee,et al.  Precise and realistic utility functions for user-centric performance analysis of schedulers , 2007, HPDC '07.

[26]  Jordi Torres,et al.  Managing SLAs of heterogeneous workloads using dynamic application placement , 2008, HPDC '08.