A detailed MPI communication model for distributed systems

Message Passing Interface (MPI) is the most popular communication interface used in today PC clusters and other cluster-type parallel/distributed computers. Up-to-date the most popular analytical MPI communication performance model for parallel/distributed machines is the LogGP model, which is mostly based on system hardware parameters. Due to the popularity of MPI, the improvements of connection network in the past few years, and the development of MPI for computational Grids, LogGP model needs to be re-evaluated for the detailed hardware performance and for the inclusion of middleware overheads for different data structures. In this article, we use our experiment results to show that the current LogGP communication model is too limited for today parallel/distributed systems. We propose a modification by including into the model important factors that have been left out. We itemize the terms of the model to show the consistency and the meaning of these communication costs, which we believe to be the starting point for modeling MPI communication cost on the Grids. In this work, we start with point-to-point communication and plan to extend to other communication patterns, such as the broadcast communication.