On the Tradeoff Between Parallelism and Communication

The problem of the optimal partitioning of a computation into parallel tasks is studied. Tasks are created dynamically and there is a cost associated with the creation and with the transfer of parameters and results. In this case, increasing the number of tasks does not always decrease the execution time. We analyze quantitatively the tradeoff between parallelism and communication. Several processor interconnection structures are considered: the ring, the tree, the binary k-cube, the complete connection. The optimal number of tasks that maximizes the speedup and the quality is given.