Customized dynamic load balancing for a network of workstations

Load balancing involves assigning to each processor work proportional to its performance, minimizing the execution time of the program. Although static load balancing can solve many problems (e.g., those caused by processor heterogeneity and non uniform loops) for most regular applications, the transient external load due to multiple users on a network of workstations necessitates a dynamic approach to load balancing. We examine the behavior of global vs. local, and centralized vs. distributed, load balancing strategies. We show that different schemes are best for different applications under varying program and system parameters. Therefore, customized load balancing schemes become essential for good performance. We present a hybrid compile time and run time modeling and decision process which selects (customizes) the best scheme, along with automatic generation of parallel code with calls to a run time library for load balancing.

[1]  Robert D. Blumofe,et al.  Scheduling large-scale parallel computations on networks of workstations , 1994, Proceedings of 3rd IEEE International Symposium on High Performance Distributed Computing.

[2]  Mohammed J. Zaki,et al.  Loop scheduling for heterogeneity , 1995, Proceedings of the Fourth IEEE International Symposium on High Performance Distributed Computing.

[3]  Francine Berman,et al.  Application-Level Scheduling on Distributed Heterogeneous Networks , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[4]  Michael J. Quinn,et al.  Data-parallel programming on a network of heterogeneous workstations , 1993, Concurr. Pract. Exp..

[5]  Evangelos P. Markatos,et al.  Using processor affinity in loop scheduling on shared-memory multiprocessors , 1992, Supercomputing '92.

[6]  Steven W. K. Tjiang,et al.  An overview of the suif compiler system , 1990 .

[7]  Anthony P. Reeves,et al.  High performance computing on a cluster of workstations , 1992, Proceedings of the First International Symposium on High-Performance Distributed Computing. (HPDC-1).

[8]  Xiaodong Zhang,et al.  Erratum: "An Effective and Practical Performance Prediction Model for Parallel Computing on Nondedicated Heterogeneous NOW" , 1997, J. Parallel Distributed Comput..

[9]  Keshav Pingali,et al.  Access normalization: loop restructuring for NUMA computers , 1993, TOCS.

[10]  Peter Steenkiste,et al.  Automatic generation of parallel programs with dynamic load balancing , 1994, Proceedings of 3rd IEEE International Symposium on High Performance Distributed Computing.

[11]  Keshav Pingali,et al.  Access normalization: loop restructuring for NUMA compilers , 1992, ASPLOS V.

[12]  Andrew S. Grimshaw,et al.  Metasystems: An Approach Combining Parallel Processing and Heterogeneous Distributed Computing Systems , 1994, J. Parallel Distributed Comput..

[13]  Wei Li,et al.  Performance Impact of Processor and Memory Heterogeneity in a Network of Machines , 1995 .

[14]  Alan Weiss,et al.  Allocating Independent Subtasks on Parallel Processors , 1985, IEEE Transactions on Software Engineering.

[15]  Robert M. Keller,et al.  The Gradient Model Load Balancing Method , 1987, IEEE Transactions on Software Engineering.

[16]  Erik Seligman,et al.  Dome: Parallel Programming in a Heteroge-neous Multi-User Environment , 1995 .

[17]  Vikram A. Saletore,et al.  Parallel computations on the CHARM heterogeneous workstation cluster , 1994, Proceedings of 3rd IEEE International Symposium on High Performance Distributed Computing.

[18]  Peter Steenkiste,et al.  A general architecture for load balancing in a distributed-memory environment , 1993, [1993] Proceedings. The 13th International Conference on Distributed Computing Systems.

[19]  CONSTANTINE D. POLYCHRONOPOULOS,et al.  Guided Self-Scheduling: A Practical Scheduling Scheme for Parallel Supercomputers , 1987, IEEE Transactions on Computers.

[20]  Constantine D. Polychronopoulos,et al.  Parallel programming and compilers , 1988 .