Performance Efiects of Scheduling Strategies for Master/Slave Distributed Applications

The achievement of parallel application performance on non-dedicated workstation clusters requires careful attention to the scheduling of tasks and communication on the underlying platform. In the literature, application scheduling policies are usually chosen by matching the resource requirements of an application with the performance characteristics of the target platform. However, when clusters of workstations are shared with other users, platform performance is non-uniform and varies over time. As a result, the performance of distinct scheduling policies may also vary depending on dynamic system state and particular characteristics of the job being run. Our experimental work focuses on a master/slave parallel ray-tracing application executing on a set of workstation clusters at UCSD and the San Diego Supercomputer Center. The experiments show that two different scheduling strategies, one static and one dynamic, exhibit very different performance sensitivities to variabilities in resource capabilities and workload distribution. We demonstrate for our example application that neither scheduling strategy by itself consistently induces the best application performance (minimal execution time) when running on the same resources under normally experienced production operating conditions. These results support the idea that dynamic selection of appropriate scheduling strategies to match run-time conditions provides a promising approach to achieving application performance for master/slave applications on heterogeneous time-shared workstation clusters.