Optimal Algorithms for Half-Duplex Inter-Group All-to-All Broadcast on Fully Connected and Ring Topologies

Half-duplex inter-group collective communications are bipartite message transfer patterns such that the processes in a sender group pass messages to the processes in a receiver group. These communication patterns serve as basic operations for scientific application workflows. In this paper, we present optimal parallel algorithms for half-duplex inter-group all-to-all broadcast under bidirectional communication constraint on fully connected and ring topologies. We implement the algorithms using MPI communication functions and perform experiments on Cori. For the fully connected topology case, we compare our algorithms with production MPI libraries. For the ring topology case, we implement our proposed algorithms using MPI_Sendrecv function to emulate a ring topology environment. The proposed algorithms are compared with the intra-group Allgather algorithm emulated under the same environment. Message sizes ranging from 32KB to 4MB are used for evaluations. The proposed algorithms for fully connected topology are up to 5 times faster than the root gathering algorithm adopted by MPICH. The proposed algorithms for the ring topology are up to 1.4 times faster than the intragroup Allgather algorithm.