A de-centralized scheduling and load balancing algorithm for heterogeneous grid environments

In the past two decades, numerous scheduling and load balancing techniques have been proposed for locally distributed multiprocessor systems. However they all suffer from significant deficiencies when extended to a Grid environment: some use a centralized approach that renders the algorithm unscalable, while others assume the overhead involved in searching for appropriate resources to be negligible. Furthermore, classical scheduling algorithms do not consider a Grid node to be N-resource rich and merely work towards maximizing the utilization of one of the resources. In this paper, we propose a new scheduling and load balancing algorithm for a generalized Grid model of N-resource nodes that not only takes into account the node and network heterogeneity, but also considers the overhead involved in coordinating among the nodes. Our algorithm is decentralized, scalable, and overlaps the node coordination time with that of the actual processing of ready jobs, thus saving valuable clock cycles needed for making decisions. The proposed algorithm is studied by conducting simulations using the Message Passing Interface (MPI) paradigm.

[1]  L.M. Ni,et al.  Trapezoid Self-Scheduling: A Practical Scheduling Scheme for Parallel Compilers , 1993, IEEE Trans. Parallel Distributed Syst..

[2]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.

[3]  Vipin Kumar,et al.  Load balancing across near-homogeneous multi-resource servers , 2000, Proceedings 9th Heterogeneous Computing Workshop (HCW 2000) (Cat. No.PR00556).

[4]  Vipin Kumar,et al.  Scalable Load Balancing Techniques for Parallel Computers , 1994, J. Parallel Distributed Comput..

[5]  Josep Torrellas,et al.  Evaluating the Performance of Cache-Affinity Scheduling in Shared-Memory Multiprocessors , 1995, J. Parallel Distributed Comput..

[6]  Laxmikant V. Kalé,et al.  Comparing the Performance of Two Dynamic Load Distribution Methods , 1988, ICPP.

[7]  V. Kumar,et al.  Job Scheduling in the presence of Multiple Resource Requirements , 1999, ACM/IEEE SC 1999 Conference (SC'99).

[8]  Jon B. Weissman,et al.  Scheduling parallel computations in a heterogeneous environment , 1996 .

[9]  J. Liu,et al.  Self-scheduling on distributed-memory machines , 1993, Supercomputing '93.

[10]  Cauligi S. Raghavendra,et al.  A Dynamic Load-Balancing Policy With a Central Job Dispatcher (LBC) , 1992, IEEE Trans. Software Eng..

[11]  Mukesh Singhal,et al.  Load distributing for locally distributed systems , 1992, Computer.

[12]  Edward D. Lazowska,et al.  A Comparison of Receiver-Initiated and Sender-Initiated Adaptive Load Sharing , 1986, Perform. Evaluation.

[13]  Anthony P. Reeves,et al.  Strategies for Dynamic Load Balancing on Highly Parallel Computers , 1993, IEEE Trans. Parallel Distributed Syst..

[14]  Yongbing Zhang,et al.  Comparison of dynamic and static load-balancing strategies in heterogeneous distributed systems , 1997 .

[15]  Salim Hariri,et al.  An agent based dynamic load balancing system , 2000, Proceedings 2000 International Workshop on Autonomous Decentralized System (Cat. No.00EX449).

[16]  William E. Johnston,et al.  Grids as production computing environments: the engineering aspects of NASA's Information Power Grid , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[17]  Songnian Zhou.  A Trace-Driven Simulation Study of Dynamic Load Balancing , 1988, IEEE Trans. Software Eng..

[18]  Ladislau Bölöni,et al.  A Comparison of Eleven Static Heuristics for Mapping a Class of Independent Tasks onto Heterogeneous Distributed Computing Systems , 2001, J. Parallel Distributed Comput..

[19]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[20]  Dong Li,et al.  A dynamic load balancing algorithm based on distributed database system , 2000, Proceedings Fourth International Conference/Exhibition on High Performance Computing in the Asia-Pacific Region.