Virtual Machine Hosting for Networked Clusters: Building the Foundations for "Autonomic" Orchestration

Virtualization technology offers powerful resource management mechanisms, including performance-isolating resource schedulers, live migration, and suspend/resume. But how should networked virtual computing systems use these mechanisms? A grand challenge is to devise practical policies to drive these mechanisms in a self-managing or "autonomic" system, without relying on human operators. This paper explores architectural and algorithmic issues for resource management policy and orchestration in Shirako, a system for on-demand leasing of shared networked resources in federated clusters. Shirako enables a flexible factoring of resource management functions across the participants in a federated system, to accommodate a range of models of distributed virtual computing. We present extensions to Shirako to provision fine-grained virtual machine "slivers" and drive virtual machine migration. We illustrate the interactions of provisioning and placement/migration policies, and their impact.

[1]  Andrew Warfield,et al.  Live migration of virtual machines , 2005, NSDI.

[2]  Jeffrey S. Chase,et al.  Making Scheduling "Cool": Temperature-Aware Workload Placement in Data Centers , 2005, USENIX Annual Technical Conference, General Track.

[3]  Amin Vahdat,et al.  SHARP: an architecture for secure resource peering , 2003, SOSP '03.

[4]  Borja Sotomayor,et al.  Virtual Clusters for Grid Communities , 2006, Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06).

[5]  Timothy Roscoe,et al.  Resource overbooking and application profiling in shared hosting platforms , 2002, OSDI '02.

[6]  Jeff Chase,et al.  Self-recharging virtual currency , 2005, P2PECON '05.

[7]  J. Chase,et al.  Data Center Workload Monitoring , Analysis , and Emulation , 2005 .

[8]  T. Kelly.  Utility-Directed Allocation , 2003 .

[9]  Wei Jin,et al.  USENIX Association Proceedings of USITS ’ 03 : 4 th USENIX Symposium on Internet Technologies and Systems , 2003 .

[10]  Carl A. Waldspurger,et al.  Memory resource management in VMware ESX server , 2002, OSDI '02.

[11]  Asser N. Tantawi,et al.  Dynamic placement for clustered web applications , 2006, WWW '06.

[12]  David E. Irwin,et al.  Dynamic virtual clusters in a grid site manager , 2003, High Performance Distributed Computing, 2003. Proceedings. 12th IEEE International Symposium on.

[13]  Amin Vahdat,et al.  Managing energy and server resources in hosting centers , 2001, SOSP.

[14]  Andrew Warfield,et al.  Xen and the art of virtualization , 2003, SOSP '03.

[15]  Terence Kelly,et al.  Generalized knapsack solvers for multi-unit combinatorial auctions: analysis and application to computational resource allocation , 2004, AAMAS'04.

[16]  L. Ramakrishnan,et al.  Toward a Doctrine of Containment: Grid Hosting with Adaptive Resource Control , 2006, ACM/IEEE SC 2006 Conference (SC'06).

[17]  David E. Irwin,et al.  Sharing Networked Resources with Brokered Leases , 2006, USENIX Annual Technical Conference, General Track.

[18]  Prashant J. Shenoy,et al.  Resource overbooking and application profiling in shared hosting platforms , 2002, OSDI '02.

[19]  Joseph Hall,et al.  On algorithms for efficient data migration , 2001, SODA '01.