Dynamic on-line task scheduling on parallel processors

Abstract The problem of scheduling on-line a stream of multi-tasked jobs on a set of parallel processors is considered. The goal is to minimize the average sojourn time experienced by each individual job in the system. The arriving jobs are comprised of parallel applications, each consisting of multiple independent tasks that are instantaneously assigned to processor queues upon its arrival to the system. The processors independently and concurrently service the tasks in their local queues which are first-come-first-served (FCFS). A job can depart from the system only when all its tasks have been executed and reassembled into the original single-job structure; until then, completed tasks are placed in a reassembly queue. All tasks have i.i.d. memoryless exponential service times and any task can be processed by any processor. Migration of tasks between queues — after their initial allocation — is not allowed. This model captures the main features of multiprocessor computer systems executing parallel programs. The key scheduling issue is as follows. When some queue backlogs are small, an incoming job should spread its tasks to those lightly loaded queues in order to take advantage of the parallel processing gain and lower its processing delay. On the other hand, when all queues are fairly congested, the job should schedule all its tasks sequentially in a single queue to avoid excessive reassembly delay (due to backlog fluctuations) and lower its task synchronization delay. In this paper, the trade-off between these two objectives is quantified and it is shown that the optimal schedule’s structure is based on backlog thresholds. Moreover, an off-line mechanism for determining these thresholds is provided, and further characteristics of the optimal scheduling policy under special cases is discussed. Finally, asymptotics and approximations for systems comprised of a large number of processors are considered.