Messages Scheduling for Data Redistribution Between Heterogeneous Clusters

In this paper, we tackle the problem of redistributing data between clusters connected by a backbone. On distributed environments, communications often take more time [12] and thus lead to worse results than on local clusters. There is therefore a strong need to optimize the time needed by communications.Indeed, when an application composed of several codes running on distant clusters is executing, data are required to be redistributed between the clusters. We propose a general solution to the problem when the platform is fully heterogeneous platforms (each node of each cluster can communicate at different speed) or when some nodes have several network interface cards. We provide an algorithm for scheduling the messages that gives a solution at most twice as long as the optimal one. Simulation results show that it is giving almost optimal schedules on large redistribution patterns, and very good results in the general case.