Dynamic Load Balancing in Distributed Systems in the Presence of Delays: A Regeneration-Theory Approach

A regeneration-theory approach is undertaken to analytically characterize the average overall completion time in a distributed system. The approach considers the heterogeneity in the processing rates of the nodes as well as the randomness in the delays imposed by the communication medium. The optimal one-shot load balancing policy is developed and subsequently extended to develop an autonomous and distributed load-balancing policy that can dynamically reallocate incoming external loads at each node. This adaptive and dynamic load balancing policy is implemented and evaluated in a two-node distributed system. The performance of the proposed dynamic load-balancing policy is compared to that of static policies as well as existing dynamic load-balancing policies by considering the average completion time per task and the system processing rate in the presence of random arrivals of the external loads.

[1]  Brian W. Kernighan,et al.  An efficient heuristic procedure for partitioning graphs , 1970, Bell Syst. Tech. J..

[2]  Van Jacobson,et al.  Congestion avoidance and control , 1988, SIGCOMM '88.

[3]  J. Liu,et al.  Self-scheduling on distributed-memory machines , 1993, Supercomputing '93.

[4]  Scott Shenker,et al.  The Optimal Control of Heterogeneous Queueing Systems: A Paradigm for Load-Sharing and Routing , 1989, IEEE Trans. Computers.

[5]  Chantal Balayer,et al.  Modeling Load Balancing inside Groups using , 1997 .

[6]  Samuel T. Chanson,et al.  Hydrodynamic Load Balancing , 1999, IEEE Trans. Parallel Distributed Syst..

[7]  George Cybenko,et al.  Dynamic Load Balancing for Distributed Memory Multiprocessors , 1989, J. Parallel Distributed Comput..

[8]  Donald L. Snyder,et al.  Random Point Processes in Time and Space , 1991 .

[9]  Edward D. Lazowska,et al.  Adaptive load sharing in homogeneous distributed systems , 1986, IEEE Transactions on Software Engineering.

[10]  Zhiling Lan,et al.  Dynamic load balancing for structured adaptive mesh refinement applications , 2001, International Conference on Parallel Processing, 2001..

[11]  Sagar Dhakal,et al.  Load balancing in distributed computing over wireless LAN: effects of network delay , 2005, IEEE Wireless Communications and Networking Conference, 2005.

[12]  Jacques M. Bahi,et al.  Dynamic load balancing and efficient load estimators for asynchronous iterative algorithms , 2005, IEEE Transactions on Parallel and Distributed Systems.

[13]  Ana Cortés,et al.  Performance comparison of dynamic load-balancing strategies for distributed computing , 1999, Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. HICSS-32. Abstracts and CD-ROM of Full Papers.

[14]  Karim Y. Kabalan,et al.  ADAPTIVE LOAD SHARING IN HETERGENEOUS SYSTEMS: POLICIES, MODIFICATIONS, AND SIMULATION , 2002 .

[15]  Chaouki T. Abdallah,et al.  Dynamic Time Delay Models for Load Balancing. Part II: A Stochastic Analysis of the Effect of Delay Uncertainty , 2004 .

[16]  Thomas L. Casavant,et al.  A Taxonomy of Scheduling in General-Purpose Distributed Computing Systems , 1988, IEEE Trans. Software Eng..

[17]  Tse-Wei Wang,et al.  Dynamic Time Delay Models for Load Balancing. Part I: Deterministic Models , 2004 .

[18]  Sagar Dhakal,et al.  Dynamical discrete-time load balancing in distributed systems in the presence of time delays , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[19]  Donald R. Jones,et al.  Distributed Computing Approach for Remote Sensing Data , 2003 .

[20]  Charles Knessl,et al.  Two Tandem Queues with General Renewal Input I: Diffusion Approximation and Integral Representations , 1999, SIAM J. Appl. Math..