Towards load balancing support for I/O-intensive parallel jobs in a cluster of workstations

While previous CPU- or memory-centric load balancing schemes are capable of achieving the effective usage of global CPU and memory resources in a cluster system, the cluster exhibits significant performance drop under I/O-intensive workload conditions due to the imbalance of I/O load. To tackle this problem, we have developed two simple yet effective I/O-aware load-balancing schemes, which make it possible to balance I/O load by assigning I/O intensive sequential and parallel jobs to nodes with light I/O loads. Moreover, the proposed schemes judiciously take into account both CPU and memory load sharing in the cluster, thereby maintaining a high performance for a wide spectrum of workload. Using a set of real I/O-intensive parallel applications in addition to synthetic parallel jobs, we show that the proposed schemes consistently outperform the existing non-I/O aware load-balancing schemes for a diverse set of workload conditions. Importantly, the performance improvement becomes much more pronounced when the applications are I/O-intensive.

[1]  Amnon Barak,et al.  The home model and competitive algorithms for load balancing in a computing cluster , 2001, Proceedings 21st International Conference on Distributed Computing Systems.

[2]  Miron Livny,et al.  Managing network resources in Condor , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.

[3]  Peter Scheuermann,et al.  File Assignment in Parallel I/O Systems with Minimal Variance of Service Time , 2000, IEEE Trans. Computers.

[4]  Andrea C. Arpaci-Dusseau,et al.  Effective distributed scheduling of parallel workloads , 1996, SIGMETRICS '96.

[5]  Jameela Al-Jaroodi,et al.  An agent-based infrastructure for parallel Java on heterogeneous clusters , 2002, Proceedings. IEEE International Conference on Cluster Computing.

[6]  Andreas Mueller,et al.  Fast sequential and parallel algorithms for association rule mining: a comparison , 1995 .

[7]  Remzi H. Arpaci-Dusseau,et al.  Storage-Aware Caching: Revisiting Caching for Heterogeneous Storage Systems , 2002, FAST.

[8]  Xiao Qin,et al.  A case study of parallel I/O for biological sequence search on Linux clusters , 2003, 2003 Proceedings IEEE International Conference on Cluster Computing.

[9]  Bruce Hendrickson,et al.  The Torus-Wrap Mapping for Dense Matrix Calculations on Massively Parallel Computers , 1994, SIAM J. Sci. Comput..

[10]  Joel H. Saltz,et al.  Titan: a high-performance remote-sensing database , 1997, Proceedings 13th International Conference on Data Engineering.

[11]  Xiao Qin,et al.  Dynamic Load Balancing for I/O- and Memory-Intensive Workload in clusters Using a Feedback Control Mechanism , 2003, Euro-Par.

[12]  Joel H. Saltz,et al.  Tuning the performance of I/O-intensive parallel applications , 1996, IOPADS '96.

[13]  Sanjeev Setia,et al.  Availability and utility of idle memory in workstation clusters , 1999, SIGMETRICS '99.

[14]  Kihong Park,et al.  Towards communication-sensitive load balancing , 2001, Proceedings 21st International Conference on Distributed Computing Systems.

[15]  Zhou Lei,et al.  The portable batch scheduler and the maui scheduler on linux clusters , 2000 .

[16]  Marianne Winslett,et al.  Faster collective output through active buffering , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[17]  Anand Sivasubramaniam,et al.  Gang Scheduling Extensions for I/O Intensive Workloads , 2003, JSSPP.

[18]  Li Xiao,et al.  Improving distributed workload performance by sharing both CPU and memory resources , 2000, Proceedings 20th IEEE International Conference on Distributed Computing Systems.

[19]  Xiao Qin,et al.  A dynamic load balancing scheme for I/O-intensive applications in distributed systems , 2003, 2003 International Conference on Parallel Processing Workshops, 2003. Proceedings..

[20]  Joel H. Saltz,et al.  Requirements of I/O systems for parallel machines: an application-driven study , 1997 .

[21]  George C. Polyzos,et al.  Dynamic I/O characterization of I/O intensive scientific applications , 1994, Proceedings of Supercomputing '94.

[22]  Xiao Qin,et al.  Dynamic Load Balancing for I/O-Intensive Tasks on Heterogeneous Clusters , 2003, HiPC.