In-Memory Checkpointing for MPI Programs by XOR-Based Double-Erasure Codes

Today, the scale of High performance computing (HPC) systems is much larger than ever. This brings a challenge to fault tolerance of HPC systems. MPI (Message Passing Interface) is one of the most important programming tools for HPC. There are quite a few fault-tolerant extensions for MPI, such as MPICH-V, StarFish, FT-MPI and so on. Most of them are based on on-disk checkpointing. In this paper, we apply two kinds of XOR-based double-erasure codes - RDP (Row-Diagonal Parity) and B-Code to in-memory checkpointing for MPI programs. We develop scalable checkpointing/recovery algorithms which embed erasure code encoding/decoding computation into MPI collective communications operations. The experiments show that the scalable algorithms decrease communication overhead and balance computation effectively. Our approach provides highly reliable, fast in-memory checkpointing for MPI programs.

[1]  Kai Li,et al.  Diskless Checkpointing , 1998, IEEE Trans. Parallel Distributed Syst..

[2]  Garth A. Gibson,et al.  RAID: high-performance, reliable secondary storage , 1994, CSUR.

[3]  George Karypis,et al.  Introduction to Parallel Computing Solution Manual , 2003 .

[4]  James S. Plank.  A tutorial on Reed-Solomon coding for fault-tolerance in RAID-like systems , 1997 .

[5]  Jack J. Dongarra,et al.  FT-MPI: Fault Tolerant MPI, Supporting Dynamic Applications in a Dynamic World , 2000, PVM/MPI.

[6]  Thomas Hérault,et al.  MPICH-V: Toward a Scalable Fault Tolerant MPI for Volatile Nodes , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[7]  J. Plank.  Optimizing Cauchy Reed-Solomon Codes for Fault-Tolerant Storage Applications , 2005 .

[8]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[9]  Georg Stellner,et al.  CoCheck: checkpointing and process migration for MPI , 1996, Proceedings of International Conference on Parallel Processing.

[10]  Zhang Yu,et al.  The Performance of Erasure Codes Used in FT-MPI , 2009, 2009 International Forum on Information Technology and Applications.

[11]  Mario Blaum.  A Family of MDS Array Codes with Minimal Number of Encoding Operations , 2006, 2006 IEEE International Symposium on Information Theory.

[12]  C. Colbourn,et al.  Handbook of Combinatorial Designs , 2006 .

[13]  James S. Plank.  The RAID-6 Liberation Codes , 2008, FAST.

[14]  Jack Dongarra,et al.  Recent Advances in Parallel Virtual Machine and Message Passing Interface, 15th European PVM/MPI Users' Group Meeting, Dublin, Ireland, September 7-10, 2008. Proceedings , 2008, PVM/MPI.

[15]  Roy Friedman,et al.  Starfish: Fault-Tolerant Dynamic MPI Programs on Clusters of Workstations , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[16]  Peter F. Corbett,et al.  Row-Diagonal Parity for Double Disk Failure Correction (Awarded Best Paper!) , 2004, USENIX Conference on File and Storage Technologies.

[17]  Peter F. Corbett,et al.  Awarded Best Paper! -- Row-Diagonal Parity for Double Disk Failure Correction , 2004 .

[18]  George Bosilca,et al.  Fault tolerant high performance computing by a coding approach , 2005, PPoPP.