Allocating Independent Tasks to Parallel Processors: An Experimental Study

We study a scheduling or allocation problem with the following characteristics: The goal is to execute a number of unspecified tasks on a parallel machine in any order and as quickly as possible. The tasks are maintained by a central monitor that will hand out batches of a variable number of tasks to requesting processors. A processor works on the batch assigned to it until it has completed all tasks in the batch, at which point it returns to the monitor for another batch. The time needed to execute a task is assumed to be a random variable with known mean and variance, and the execution times of distinct tasks are assumed to be independent. Moreover, each time a processor obtains a new batch from the monitor, it suffers a fixed delay. The challenge is to allocate batches to processors in such a way as to achieve a small expected overall finishing time. We introduce a new allocation strategy, the Bold strategy, and show that it outperforms other strategies suggested in the literature in a number of simulations.

[1]  Jeanette P. Schmidt,et al.  Load-sharing in heterogeneous systems via weighted factoring , 1996, SPAA '96.

[2]  Alastair J. Walker,et al.  An Efficient Method for Generating Discrete Random Variables with General Distributions , 1977, TOMS.

[3]  R. A. Doney,et al.  4. Probability and Random Processes , 1993 .

[4]  Linus Schrage,et al.  A guide to simulation , 1983 .

[5]  Paul Bratley,et al.  A guide to simulation , 1983 .

[6]  L. Devroye Non-Uniform Random Variate Generation , 1986 .

[7]  H. O. Hartley,et al.  Universal Bounds for Mean Range and Extreme Observation , 1954 .

[8]  E. J. Gumbel,et al.  The Maxima of the Mean Largest Value and of the Range , 1954 .

[9]  Steven Lucco,et al.  Orchestrating interactions among parallel computations , 1993, PLDI '93.

[10]  Kurt Mehlhorn,et al.  LEDA: a platform for combinatorial and geometric computing , 1997, CACM.

[11]  R. Durrett Probability: Theory and Examples , 1993 .

[12]  CONSTANTINE D. POLYCHRONOPOULOS,et al.  Guided Self-Scheduling: A Practical Scheduling Scheme for Parallel Supercomputers , 1987, IEEE Transactions on Computers.

[13]  Alan Weiss,et al.  Allocating Independent Subtasks on Parallel Processors , 1985, IEEE Transactions on Software Engineering.

[14]  Edith Schonberg,et al.  Factoring: a method for scheduling parallel loops , 1992 .

[15]  N. L. Johnson,et al.  Continuous Univariate Distributions. , 1995 .

[16]  I. Banicescu,et al.  Balancing Processor Loads and Exploiting Data Locality in N-Body Simulations , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[17]  Allan Gut,et al.  An intermediate course in probability , 1995 .

[18]  R. Cheng,et al.  The Generation of Gamma Variables with Non‐Integral Shape Parameter , 1977 .

[19]  Steven Lucco,et al.  A dynamic scheduling method for irregular parallel programs , 1992, PLDI '92.

[20]  Catherine C. McGeoch Feature Article - Toward an Experimental Method for Algorithm Simulation , 1996, INFORMS J. Comput..