VMckpt: lightweight and live virtual machine checkpointing

Recent advance of virtualization technology provides a new approach to check-point/restart at the virtual machine (VM) level. In contrast to traditional process-level checkpointing, checkpointing at the virtualization layer brings up several advantages, such as compatibility, transparence, flexibility and simplicity. However, because the virtualization layer has little semantic knowledge about the operation system and the applications running atop, VM-layer checkpointing requires saving the entire operating system state rather than a single process. The overhead may render the approach impractical. To reduce the size of VM checkpoint, in this paper we propose a page eviction scheme and an incremental checkpointing mechanism to avoid saving unnecessary VM pages in the checkpoint. To keep the system online transparently, we propose a live checkpointing mechanism by saving the memory image in a copy-on-write (COW) manner. We implement the performance optimization mechanisms in a prototype system, called VMckpt. Experimental results with a group of representative applications show that our page eviction scheme and incremental checkpointing can significantly reduce the checkpoint file size by up to 87% and shorten the total checkpointing/restart time by a factor of up to 71%, in comparison with the Xens default checkpointing mechanism. The observed application downtimes due to checkpointing can be reduced to as small as 300 ms.

[1]  F. Cappello,et al.  Blocking vs. Non-Blocking Coordinated Checkpointing for Large-Scale Fault Tolerant MPI , 2006, ACM/IEEE SC 2006 Conference (SC'06).

[2]  Kai Li,et al.  CLIP: A Checkpointing Tool for Message Passing Parallel Programs , 1997, ACM/IEEE SC 1997 Conference (SC'97).

[3]  Andrew Warfield,et al.  Live migration of virtual machines , 2005, NSDI.

[4]  A WaldspurgerCarl Memory resource management in VMware ESX server , 2002 .

[5]  Kai Li,et al.  Memory Exclusion: Optimizing the Performance of Checkpointing Systems , 1999, Softw. Pract. Exp..

[6]  Cheng-Zhong Xu,et al.  Exploring event correlation for failure prediction in coalitions of clusters , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[7]  Hai Jin,et al.  Live Virtual Machine Migration via Asynchronous Replication and State Synchronization , 2011, IEEE Transactions on Parallel and Distributed Systems.

[8]  Song Jiang,et al.  Transparent, Incremental Checkpointing at Kernel Level: a Foundation for Fault Tolerance for Parallel Computers , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[9]  Geoffroy Vallée,et al.  Checkpoint/Restart of Virtual Machines Based on Xen , 2006 .

[10]  Kai Li,et al.  Memory exclusion: optimizing the performance of checkpointing systems , 1999 .

[11]  Dutch T. Meyer,et al.  Parallax: virtual disks for virtual machines , 2008, Eurosys '08.

[12]  Patrick Th. Eugster,et al.  VNsnap: Taking snapshots of virtual networked environments with minimal downtime , 2009, 2009 IEEE/IFIP International Conference on Dependable Systems & Networks.

[13]  Kartik Gopalan,et al.  Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning , 2009, VEE '09.

[14]  M. Rosenblum,et al.  Optimizing the migration of virtual computers , 2002, OSDI '02.

[15]  Thomas Hérault,et al.  Blocking vs. non-blocking coordinated checkpointing for large-scale fault tolerant MPI Protocols , 2008, Future Gener. Comput. Syst..

[16]  Carl A. Waldspurger,et al.  Memory resource management in VMware ESX server , 2002, OSDI '02.

[17]  Hai Jin,et al.  Optimize Performance of Virtual Machine Checkpointing via Memory Exclusion , 2009, 2009 Fourth ChinaGrid Annual Conference.

[18]  Samuel T. King,et al.  Debugging Operating Systems with Time-Traveling Virtual Machines (Awarded General Track Best Paper Award!) , 2005, USENIX Annual Technical Conference, General Track.

[19]  Dutch T. Meyer,et al.  Remus: High Availability via Asynchronous Virtual Machine Replication. (Best Paper) , 2008, NSDI.

[20]  Muli Ben-Yehuda,et al.  Virtual machine time travel using continuous data protection and checkpointing , 2008, OPSR.

[21]  Kai Li,et al.  Libckpt: Transparent Checkpointing under UNIX , 1995, USENIX.

[22]  Christian Engelmann,et al.  Proactive fault tolerance for HPC with Xen virtualization , 2007, ICS '07.

[23]  Borja Sotomayor,et al.  Combining batch execution and leasing using virtual machines , 2008, HPDC '08.

[24]  Daniel Marques,et al.  Optimizing checkpoint sizes in the C3 system , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[25]  Song Jiang,et al.  Current practice and a direction forward in checkpoint/restart implementations for fault tolerance , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[26]  Wenguang Chen,et al.  CprFS: a user-level file system to support consistent file states for checkpoint and restart , 2008, ICS '08.