Minimizing communication conflicts with load-skewing task assignment techniques on network of workstations

In a bus-based network of workstations, a perfectly load balance arrangement does not always lead to the best performance due to potential communication resource conflicts. Such a situation arises when work-stations tend to compete for the shared bus after they all finish their assigned workload at about the same time under such a load arrangement. In this paper, we provide a thorough analysis on how such communication conflicts can be minimized in a bus-based system by using a load-skewing assignment method. A probabilistic model is used to analyze the needed skewing factor for a one-stage divide-and-conquer programming paradigm. Our analytical results are closely confirmed by various simulation and experiment outcome. The proposed technique can also be extended to handle multi-stage cases.

[1]  Bo Yang,et al.  Load balancing technique for parallel search with statistical model , 1995, Proceedings International Phoenix Conference on Computers and Communications.

[2]  Alan Weiss,et al.  Allocating Independent Subtasks on Parallel Processors , 1985, IEEE Transactions on Software Engineering.

[3]  Robert D. Blumofe,et al.  Scheduling multithreaded computations by work stealing , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[4]  Mary K. Vernon,et al.  The influence of random delays on parallel execution times , 1993, SIGMETRICS '93.

[5]  Jon G. Kuhl,et al.  Stochastic Performance Models of Parallel Task Systems , 1994, SIGMETRICS.

[6]  Thomas L. Casavant,et al.  A Taxonomy of Scheduling in General-Purpose Distributed Computing Systems , 1988, IEEE Trans. Software Eng..

[7]  H. O. Hartley,et al.  Universal Bounds for Mean Range and Extreme Observation , 1954 .

[8]  Frank Thomson Leighton,et al.  Improved methods for hiding latency in high bandwidth networks (extended abstract) , 1996, SPAA '96.

[9]  Edward D. Lazowska,et al.  Adaptive load sharing in homogeneous distributed systems , 1986, IEEE Transactions on Software Engineering.

[10]  Dan C. Marinescu,et al.  Synchronization and load imbalance effects in distributed memory multi-processor systems , 1991, Concurr. Pract. Exp..

[11]  Peter Steenkiste,et al.  A general architecture for load balancing in a distributed-memory environment , 1993, [1993] Proceedings. The 13th International Conference on Distributed Computing Systems.

[12]  Edith Schonberg,et al.  Factoring: a method for scheduling parallel loops , 1992 .

[13]  Vipin Kumar,et al.  On the Efficiency of Parallel Backtracking , 1993, IEEE Trans. Parallel Distributed Syst..

[14]  Linda F. Wilson,et al.  Synchronization and communication in algorithmic structures , 1994, Proceedings of 1994 6th IEEE Symposium on Parallel and Distributed Processing.

[15]  Edith Schonberg,et al.  Factoring: a practical and robust method for scheduling parallel loops , 1991, Proceedings of the 1991 ACM/IEEE Conference on Supercomputing (Supercomputing '91).

[16]  Mukesh Singhal,et al.  Load distributing for locally distributed systems , 1992, Computer.

[17]  S. Madala,et al.  Performance of Synchronous Parallel Algorithms with Regular Structures , 1991, IEEE Trans. Parallel Distributed Syst..

[18]  Michael Schmidt-Voigt.  Efficient Parallel Communication with the nCUBE 2S Processor , 1994, Parallel Comput..

[19]  Tahir M. Afzal,et al.  Load sharing in distributed computer systems , 1987 .

[20]  Lewis W. Tucker,et al.  CMMD: Active Messages on the CM-5 , 1994, Parallel Comput..

[21]  Eli Upfal,et al.  A simple load balancing scheme for task allocation in parallel machines , 1991, SPAA '91.