Server-side scheduling in cluster parallel I/O systems

Parallel I/O has become a necessity in the face of performance improvements in other areas of computing systems. Studies have shown that peak performance is infrequently realized, and work in parallel I/O optimization strives to achieve peak performance for appli- cations. In this paper we revisit one area of performance optimization in parallel I/O, that of server-side scheduling of service. With the wide variety of systems and workloads seen to- day, multiple server-side scheduling algorithms are necessary to match potential workloads. We show through experimentation that performance gains can be seen in practice through the use of alternative scheduling algorithms, but that no single algorithm provides the best perfor- mance across the board. Finally we discuss the potential for automatic matching of server-side scheduling algorithms to workloads in real-time.

[1]  Robert B. Ross,et al.  REACTIVE SCHEDULING FOR PARALLEL I/O SYSTEMS , 2000 .

[2]  Robert Geist,et al.  Disk scheduling revisited: can O(n2) algorithms compete? , 1997, ACM-SE 35.

[3]  Robert B. Ross,et al.  PVFS: A Parallel File System for Linux Clusters , 2000, Annual Linux Showcase & Conference.

[4]  Alok N. Choudhary,et al.  Design and evaluation of primitives for parallel I/O , 1993, Supercomputing '93. Proceedings.

[5]  Daniel A. Reed,et al.  Input/output access pattern classification using hidden Markov models , 1997, IOPADS '97.

[6]  David Kotz,et al.  File-system workload on a scientific multiprocessor , 1995, IEEE Parallel Distributed Technol. Syst. Appl..

[7]  T.M. Madhyastha,et al.  Intelligent, adaptive file system policy selection , 1996, Proceedings of 6th Symposium on the Frontiers of Massively Parallel Computation (Frontiers '96).

[8]  John Krystynak,et al.  Performance characteristics of the iPSC/860 and CM-2 I/O systems , 1993, [1993] Proceedings Seventh International Parallel Processing Symposium.

[9]  David Kotz,et al.  Disk-directed I/O for MIMD multiprocessors , 1994, OSDI '94.

[10]  Rajeev Thakur,et al.  Users guide for ROMIO: A high-performance, portable MPI-IO implementation , 1997 .

[11]  Robert Ross,et al.  Implementation and performance of a parallel file system for high performance distributed applications , 1996, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing.

[12]  Peter J. Denning,et al.  Effects of scheduling on file memory operations , 1899, AFIPS '67 (Spring).

[13]  Rakesh Krishnaiyer,et al.  PASSION: Parallel And Scalable Software for Input-Output , 1994 .

[14]  Gil Utard,et al.  MPI-IO on a parallel file system for cluster of workstations , 1999, ICWC 99. IEEE Computer Society International Workshop on Cluster Computing.

[15]  Marianne Winslett,et al.  Server-Directed Collective I/O in Panda , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[16]  P. Merkey,et al.  Beowulf: harnessing the power of parallelism in a pile-of-PCs , 1997, 1997 IEEE Aerospace Conference.