LA-MPI : The Design and Implementation of a Network-Fault-Tolerant MPI for Terascale Clusters

In this paper we discuss unique architectural elements of the Los Alamos Message Passing Interface (LA-MPI). LAMPI is a high-performance, network fault-tolerant, threadsafe MPI library designed for terascale clusters that are inherently unreliable due to their sheer number of system components and inherent trade-offs between cost and performance. We examine in detail the design concepts used to implement LA-MPI. These include reliability features, such as application-level checksumming, message retransmission, and automatic message re-routing. Other key performance enhancing features, such as concurrent message routing over multiple, diverse network adapters and protocols, and communication-specific optimizations (e.g., shared memory) are examined. Email: lampi-support@lanl.gov. Los Alamos report LAUR-03-0939. Los Alamos National Laboratory is operated by the University of California for the National Nuclear Security Administration of the United States Department of Energy under contract W-7405-ENG-36. Project support was provided through ASCI/PSE and the Los Alamos Computer Science Institute.