Self-Adjusting Scheduling of Master-Worker Applications on Distributed Clusters

Strategies for scheduling parallel applications on a distributed system must trade-off processor application speed-up and resource efficiency. Most existing strategies focus mainly on achieving high application speed-up without taking into account the efficiency factor. This paper presents our experiences with a self-adaptive scheduling strategy that dynamically adjusts the number of resources used by an application based on performance measures gathered during its execution. The strategy seeks to maximize resource efficiency while minimizing the impact in loss of speedup. It also uses the measured times to decide how to assign tasks to resources. This work has been carried out in the context of opportunistic clusters of machines and we report the results achieved by our strategy when it was applied to an image thinning application run on a Condor pool.

[1]  Rajesh Raman,et al.  High Throughput Monte Carlo , 1999, PPSC.

[2]  Rajkumar Buyya,et al.  High Performance Cluster Computing: Architectures and Systems , 1999 .

[3]  David Abramson,et al.  Nimrod: a tool for performing parametrised simulations using distributed workstations , 1995, Proceedings of the Fourth IEEE International Symposium on High Performance Distributed Computing.

[4]  Zicheng Guo,et al.  Fast fully parallel thinning algorithms , 1991, CVGIP Image Underst..

[5]  Rajkumar Buyya,et al.  Parallel Programming Models and Paradigms , 1998 .

[6]  Jeff T. Linderoth,et al.  An enabling framework for master-worker applications on the Computational Grid , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.

[7]  Miron Livny,et al.  Interfacing Condor and PVM to harness the cycles of workstation clusters , 1996, Future Gener. Comput. Syst..

[8]  Miron Livny,et al.  Adaptive Scheduling for Master-Worker Applications on the Computational Grid , 2000, GRID.

[9]  Miron Livny,et al.  Evaluation of an Adaptive Scheduling Strategy for Master-Worker Applications on Clusters of Workstations , 2000, HiPC.

[10]  Henri Casanova,et al.  Adaptive Scheduling for Task Farming with Grid Middleware , 1999, Int. J. High Perform. Comput. Appl..