Dynamic Scheduling Strategies for an Adaptive, Asynchronous Parallel Global Optimization Algorithm ; CU-CS-625-92

Abstract : This paper explores the use of dynamic scheduling strategies for irregular parallel algorithms in distributed memory computational environments. The target application we consider is an adaptive asynchronous parallel algorithm with irregular structure that is used to solve the global optimization problem. In this algorithm the number of tasks and their sizes may change dynamically, so that dynamic scheduling is needed to insure that the workload is evenly distributed across the processors. We consider three dynamic scheduling strategies for implementing this algorithm: centralized scheduling, which uses a master-slave approach; distributed scheduling, which uses local information about processor workload to determine when tasks should be requested from or sent to other processors; and a new hybrid approach that we refer to as centralized mediation, that uses aspects of both centralized and distributed scheduling. The implementation of the global optimization algorithm using the scheduling strategies is discussed, and their performance is thoroughly assessed through a combination of analytic modeling, simulation, and distributed implementation. In these performance studies, the centralized mediation strategy often exhibits the best performance for both different numbers of processors and different loading conditions.