A Practical Approach to Dynamic Load Balancing

This paper presents a cohesive, practical load balancing framework that improves upon existing strategies. These techniques are portable to a broad range of prevalent architectures, including massively parallel machines, such as the Cray T3D/E and Intel Paragon, shared memory systems, such as the Silicon Graphics PowerChallenge, and networks of workstations. As part of the work, an adaptive heat diffusion scheme is presented, as well as a task selection mechanism that can preserve or improve communication locality. Unlike many previous efforts in this arena, the techniques have been applied to two large-scale industrial applications on a variety of multicomputers. In the process, this work exposes a serious deficiency in current load balancing strategies, motivating further work in this area.

[1]  George Cybenko,et al.  Dynamic Load Balancing for Distributed Memory Multiprocessors , 1989, J. Parallel Distributed Comput..

[2]  Stephen Taylor,et al.  A message‐driven programming system for fine‐grain multicomputers , 1994, Softw. Pract. Exp..

[3]  Francis C. M. Lau,et al.  Load balancing in parallel computers - theory and practice , 1996, The Kluwer international series in engineering and computer science.

[4]  Martin Berzins,et al.  Dynamic load-balancing for PDE solvers on adaptive unstructured meshes , 1995, Concurr. Pract. Exp..

[5]  Anthony P. Reeves,et al.  Strategies for Dynamic Load Balancing on Highly Parallel Computers , 1993, IEEE Trans. Parallel Distributed Syst..

[6]  Marc Rieffel,et al.  A Load Balancing Technique for Multiphase Computations , 1997 .

[7]  Carl Kesselman,et al.  Generalized communicators in the Message Passing Interface , 1996, Proceedings. Second MPI Developer's Conference.

[8]  Stephen Taylor,et al.  Concurrent Simulation of Plasma Reactors , 1997 .

[9]  Viktor K. Decyk,et al.  Dynamic load balancing for a 2D concurrent plasma PIC code , 1993 .

[10]  H. Keller,et al.  Analysis of Numerical Methods , 1969 .

[11]  Christos H. Papadimitriou,et al.  Computational complexity , 1993 .

[12]  Roy D. Williams,et al.  Performance of dynamic load balancing algorithms for unstructured mesh calculations , 1991, Concurr. Pract. Exp..

[13]  Begnaud Francis Hildebrand,et al.  Introduction to numerical analysis: 2nd edition , 1987 .

[14]  J. Watts,et al.  Parallel DSMC strategies for 3D computations , 1996, Parallel CFD.

[15]  Gene H. Golub,et al.  Matrix computations , 1983 .

[16]  Bradley C. Kuszmaul,et al.  Cilk: an efficient multithreaded runtime system , 1995, PPOPP '95.

[17]  J. Song.  A partially asynchronous and iterative algorithm for distributed load balancing , 1993, [1993] Proceedings Seventh International Parallel Processing Symposium.

[18]  Ed Zaluska,et al.  Parallel Load-Balancing: An Extension to the Gradient Model , 1995, Parallel Comput..

[19]  Bradley C. Kuszmaul,et al.  Cilk: an efficient multithreaded runtime system , 1995, PPOPP '95.

[20]  Robert M. Keller,et al.  The Gradient Model Load Balancing Method , 1987, IEEE Transactions on Software Engineering.

[21]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[22]  Jack Dongarra,et al.  MPI: The Complete Reference , 1996 .

[23]  H. Peter Hofstee,et al.  A Distributed Implementation of a Task Pool , 1991, Research Directions in High-Level Parallel Programming Languages.

[24]  Gregory Allen Kohring.  Dynamic Load Balancing for Parallelized Particle Simulations on MIMD Computers , 1995, Parallel Comput..

[25]  Daniel E. Hastings,et al.  Three-dimensional plasma particle-in-cell calculations of ion thruster backflow contamination , 1996 .

[26]  Joel H. Saltz,et al.  Runtime and language support for compiling adaptive irregular programs on distributed‐memory machines , 1995, Softw. Pract. Exp..

[27]  Jonathan Walpole,et al.  Adaptive load migration systems for PVM , 1994, Proceedings of Supercomputing '94.

[28]  David J. Evans,et al.  Dynamic Load Balancing Using Task-Transfer Probabilities , 1993, Parallel Comput..

[29]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[30]  Stephen Taylor,et al.  The concurrent graph: basic technology for irregular problems , 1996, IEEE Parallel Distributed Technol. Syst. Appl..

[31]  Dirk Roose,et al.  An Improved Spectral Bisection Algorithm and its Application to Dynamic Load Balancing , 1995, EUROSIM International Conference.

[32]  G. Carrier,et al.  Partial Differential Equations: Theory and Technique , 1976 .

[33]  G. Horton.  A Multi-Level Diffusion Method for Dynamic Load Balancing , 1993, Parallel Comput..

[34]  Marc A. Rieffel.  Concurrent Simulations of Plasma Reactors for VLSI Manufacturing , 1995 .

[35]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[36]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[37]  Alan Heirich.  Scalable Load Balancing by Diffusion , 1994 .

[38]  Samanta Roy,et al.  Numerical simulation of ion thruster plume backflow for spacecraft contamination assessment , 1995 .

[39]  Horst D. Simon,et al.  Fast multilevel implementation of recursive spectral bisection for partitioning unstructured problems , 1994, Concurr. Pract. Exp..