Homework for the Scheduling class

Overlapping communication with computation is a well-known technique to increase application performance. While it is commonly assumed that communication and computation can be overlapped at no cost, in reality they interfere with each other. In this paper we empirically evaluate the interference rate of communication on computation via measurements on a single processor communicating on a heterogeneous collection of local and remote processors, in both Java and C. We then present a model of interference, which can be used for more effective application scheduling, as demonstrated by real-world experiments.

[1]  Ramesh Subramonian,et al.  LogP: towards a realistic model of parallel computation , 1993, PPOPP '93.

[2]  Larry Carter,et al.  Autonomous protocols for bandwidth-centric scheduling of independent-task applications , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[3]  Denis Caromel,et al.  Optimizing Metacomputing with Communication-Computation Overlap , 2001, PaCT.

[4]  Larry Carter,et al.  On the interference of communication on computation in Java , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[5]  Larry Carter,et al.  Bandwidth-centric allocation of independent tasks on heterogeneous platforms , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[6]  Sandeep K. S. Gupta,et al.  A technique for overlapping computation and communication for block recursive algorithms , 1998 .

[7]  Larry Carter,et al.  Dynamic autonomous scheduling on heterogeneous systems , 2003 .

[8]  Paul D. Gader,et al.  Image algebra techniques for parallel image processing , 1987 .

[9]  Pierre Ramet,et al.  Optimal Grain Size Computation for Pipelined Algorithms , 1996, Euro-Par, Vol. I.

[10]  Philip J. Hatcher,et al.  On the Utility of Communication-Computation Overlap in Data-Parallel Programs , 1996, J. Parallel Distributed Comput..

[11]  Larry Carter,et al.  On the Parallel Execution Time of Tiled Loops , 2003, IEEE Trans. Parallel Distributed Syst..

[12]  Yves Robert,et al.  Evaluating Array Expressions On Massively Parallel Machines With Communication/ Computation Overlap , 1995, Int. J. High Perform. Comput. Appl..

[13]  Mitsuhisa Sato,et al.  Identifying the capability of overlapping computation with communication , 1996, Proceedings of the 1996 Conference on Parallel Architectures and Compilation Technique.

[14]  Prithviraj Banerjee,et al.  Techniques to overlap computation and communication in irregular iterative applications , 1994, ICS '94.

[15]  Scott B. Baden,et al.  Communication overlap in multi-tier parallel algorithms , 1998, Proceedings of the IEEE/ACM SC98 Conference.

[16]  Nectarios Koziris,et al.  Minimizing completion time for loop tiling with computation and communication overlapping , 2001, Proceedings 15th International Parallel and Distributed Processing Symposium. IPDPS 2001.

[17]  Rupak Biswas,et al.  Communication Studies of DMP and SMP Machines , 1997 .