The Design and Implementation of Message Passing Services for the BlueGene / L Supercomputer

The BlueGene/L supercoputer, with 65,536 dual-processor compute nodes, was designed from the group up to support ef£ceint execution of massively parallel message passing programs. Part of this support is an optimized implementation of MPI that leverages the hardware features of BlueGene/L. MPI for BlueGene/L is implemented on top of a more basic message-passing infrastructure called the message layer. This message layer can be used both to implement other higher-level libraries and directly by applications. MPI and the message layer are used in the two modes of operation of BlueGene/L: coprocessor mode and virtual node mode. Performance measurements show that our message-passing services deliver performance close to the hardware limits of the machine. They also show that dedicating one of the processors of a node to communication functions (coprocessor mode) greatly improves the message-passing bandwidth, whereas running two processes per compute node (virtual node mode) can have a positive impact on application performance.

[1]  David H. Bailey,et al.  The Nas Parallel Benchmarks , 1991, Int. J. High Perform. Comput. Appl..

[2]  Seth Copen Goldstein,et al.  Active Messages: A Mechanism for Integrated Communication and Computation , 1992, [1992] Proceedings the 19th Annual International Symposium on Computer Architecture.

[3]  Scott Pakin,et al.  High Performance Messaging on Workstations: Illinois Fast Messages (FM) for Myrinet , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[4]  Thorsten von Eicken,et al.  U-Net: a user-level network interface for parallel and distributed computing , 1995, SOSP.

[5]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[6]  Giovanni Chiola,et al.  GAMMA: A low-cost network of workstations based on active messages , 1997, PDP.

[7]  Burkhard D. Steinmacher-Burow,et al.  Cellular supercomputing with system-on-a-chip , 2002, 2002 IEEE International Solid-State Circuits Conference. Digest of Technical Papers (Cat. No.02CH37315).

[8]  David F. Heidel,et al.  An Overview of the BlueGene/L Supercomputer , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[9]  William Gropp,et al.  MPI on BlueGene/L: Designing an Efficient General Purpose Messaging Solution for a Large Cellular System , 2003, PVM/MPI.

[10]  Dave Turner,et al.  Integrating New Capabilities into NetPIPE , 2003, PVM/MPI.

[11]  José E. Moreira,et al.  An Overview Of The Bluegene/L System Software Organization , 2003, Parallel Process. Lett..