Dynamic Scheduling of Virtual Machines Running HPC Workloads in Scientific Grids

The primary motivation for enterprises to adopt virtualization technologies is to create a more agile and dynamic IT infrastructure -- with server consolidation, high resource utilization, the ability to quickly add and adjust capacity on demand -- while lowering total cost of ownership and responding more effectively to changing business conditions. However, effective management of virtualized IT environments introduces new and unique requirements, such as dynamically resizing and migrating virtual machines (VMs) in response to changing application demands. Such capacity management methods should work in conjunction with the underlying resource management mechanisms. In general, resource multiplexing and scheduling among virtual machines is poorly understood. CPU scheduling for virtual machines, for instance, has largely been borrowed from the process scheduling research in operating systems. However, it is not clear whether a straight-forward port of process schedulers to VM schedulers would perform just as well. We use the open source Xen virtual machine monitor to perform a comparative evaluation of three different CPU schedulers for virtual machines. We analyze the impact of the choice of scheduler and its parameters on application performance, and discuss challenges in estimating the application resource requirements in virtualized environments.

[1]  Renato J. O. Figueiredo,et al.  A case for grid computing on virtual machines , 2003, 23rd International Conference on Distributed Computing Systems, 2003. Proceedings..

[2]  Harrick M. Vin,et al.  A hierarchial CPU scheduler for multimedia operating systems , 1996, OSDI '96.

[3]  T Maeno,et al.  PanDA: distributed production and distributed analysis system for ATLAS , 2008 .

[4]  Gil Neiger,et al.  Intel virtualization technology , 2005, Computer.

[5]  Judy Kay,et al.  A fair share scheduler , 1988, CACM.

[6]  Hussein M. Abdel-Wahab,et al.  A proportional share resource allocation algorithm for real-time, time-shared systems , 1996, 17th IEEE Real-Time Systems Symposium.

[7]  Ludmila Cherkasova,et al.  XenMon: QoS Monitoring and Performance Profiling Tool , 2005 .

[8]  Kenneth J. Duda,et al.  Borrowed-virtual-time (BVT) scheduling: supporting latency-sensitive threads in a general-purpose scheduler , 2000, OPSR.

[9]  Harvey M. Deitel,et al.  An introduction to operating systems , 1984 .

[10]  Xianghua Xu,et al.  Performance Evaluation of the CPU Scheduler in XEN , 2008, 2008 International Symposium on Information Science and Engineering.

[11]  Carl A. Waldspurger,et al.  Lottery and stride scheduling: flexible proportional-share resource management , 1995 .

[12]  Samuel J. Leffler,et al.  The design and implementation of the 4.3 BSD Unix operating system , 1991, Addison-Wesley series in computer science.

[13]  G. J. Henry,et al.  The UNIX system: The fair share scheduler , 1984, AT&T Bell Laboratories Technical Journal.

[14]  Andrew Warfield,et al.  Reconstructing I/O , 2004 .

[15]  Harvey M. Deitel,et al.  An introduction to operating systems (2. ed.) , 1990 .

[16]  Robin Fairbairns,et al.  The Design and Implementation of an Operating System to Support Distributed Multimedia Applications , 1996, IEEE J. Sel. Areas Commun..

[17]  T. Gonen,et al.  Questions , 1927, Journal of Family Planning and Reproductive Health Care.

[18]  Tal Garfinkel,et al.  When Virtual Is Harder than Real: Security Challenges in Virtual Machine Based Computing Environments , 2005, HotOS.

[19]  Tal Garfinkel,et al.  Virtual machine monitors: current technology and future trends , 2005, Computer.

[20]  Joseph L. Hellerstein,et al.  Achieving Service Rate Objectives with Decay Usage Scheduling , 1993, IEEE Trans. Software Eng..

[21]  Peter A. Dinda,et al.  VSched: Mixing Batch And Interactive Virtual Machines Using Periodic Real-time Scheduling , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[22]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[23]  Ludmila Cherkasova,et al.  Measuring CPU Overhead for I/O Processing in the Xen Virtual Machine Monitor , 2005, USENIX ATC, General Track.

[24]  Jason Nieh,et al.  A SMART scheduler for multimedia applications , 2003, TOCS.

[25]  Paul Nilsson,et al.  Experience from a pilot based system for ATLAS , 2008 .

[26]  Amin Vahdat,et al.  Enforcing Performance Isolation Across Virtual Machines in Xen , 2006, Middleware.