Practical resilient cases for FA-MPI, a transactional fault-tolerant MPI

MPI is insufficient when confronting failures. FA-MPI (Fault-Aware MPI) provides extensions to the MPI standard designed to enable data-parallel applications to achieve resilience without sacrificing scalability. FA-MPI introduces transactions as a novel extension to the MPI message-passing model. Transactions support failure detection, isolation, mitigation, and recovery via application-driven policies. To achieve maximum achievable performance of modern machines, overlapping communication and I/O with computation through non-blocking operations is of growing importance. Therefore, we emphasize fault-tolerant, non-blocking communication operations plus a set of nestable lightweight transactional TryBlock API extensions able to exploit system and application hierarchy. This strategy enables applications to run to completion with higher probability than nominally. We modified two proxy applications---MiniFE and LULESH---by adding FA-MPI semantics to them. Finally we present performance and overhead results for 1K MPI processes.