The galley parallel file system

Most current multiprocessor file systems are designed to use multiple disks in parallel, using the high aggregate bandwidth to meet the growing I/O requirements of parallel scientific applications. Many multiprocessor file systems provide applications with a conventional Unix-like interface, allowing the application to access multiple disks transparently. This interface conceals the parallelism within the file system, increasing the ease of programmability, but making it difficult or impossible for sophisticated programmers and libraries to use knowledge about their I/O needs to exploit that parallelism. In addition to providing an insufficient interface, most current multiprocessor file systems are optimized for a different workload than they are being asked to support. We introduce Galley, a new parallel file system that is intended to efficiently support realistic scientific multiprocessor workloads. We discuss Galley''s file structure and application interface, as well as the performance advantages offered by that interface.

[1]  Carla Schlatter Ellis,et al.  Characterizing parallel file-access patterns on a large-scale multiprocessor , 1995, IPPS.

[2]  David Kotz,et al.  Performance of the Galley Parallel File System , 1996 .

[3]  James C. French,et al.  Breaking the I/O Bottleneck at the National Radio Astronomy Observatory (NRAO) , 1994 .

[4]  Richard Wheeler,et al.  it/sfs: A Parallel File System for the CM-5 , 1993, USENIX Summer.

[5]  Hoevel.  Proceedings of the twenty-first annual Hawaii international conference on system sciences , 1988 .

[6]  Marianne Winslett,et al.  Server-Directed Collective I/O in Panda , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[7]  John Wilkes,et al.  An introduction to disk drive modeling , 1994, Computer.

[8]  David Kotz,et al.  A Detailed Simulation Model of the HP 97560 Disk Drive , 1994 .

[9]  M. Winslett,et al.  A data management approach for handling large compressed arrays in high performance computing , 1995, Proceedings Frontiers '95. The Fifth Symposium on the Frontiers of Massively Parallel Computation.

[10]  Thomas H. Cormen,et al.  ViC*: A Preprocessor for Virtual-Memory C* , 1994 .

[11]  David Kotz,et al.  Performance of the gallery parallel file system , 1996, IOPADS '96.

[12]  Nils A. Nieukwejaar.  Galley: a new parallel file system for scientific applications , 1997 .

[13]  M. Winslett,et al.  Server-directed collective I/O in Panda , 1995 .

[14]  Nils Nieuwejaar,et al.  Galley: A New Parallel File System for Parallel Applications , 1996 .

[15]  Sivan Toledo,et al.  The design and implementation of SOLAR, a portable library for scalable out-of-core linear algebra computations , 1996, IOPADS '96.

[16]  James C. French,et al.  High performance access to radio astronomy data: a case study , 1994, Seventh International Working Conference on Scientific and Statistical Database Management.

[17]  David Kotz,et al.  Dynamic file-access characteristics of a production parallel scientific workload , 1994, Proceedings of Supercomputing '94.

[18]  Yarsun Hsu,et al.  Performance evaluation of a parallel I/O architecture , 1995, ICS '95.

[19]  Dror G. Feitelson,et al.  Parallel File Systems for the IBM SP Computers , 1995, IBM Syst. J..

[20]  David Kotz,et al.  Disk-directed I/O for MIMD multiprocessors , 1994, OSDI '94.

[21]  Margo I. Seltzer,et al.  Disk Scheduling Revisited , 1990 .

[22]  Andrew A. Chien,et al.  PPFS: a high performance portable parallel file system , 1995, ICS '95.

[23]  Rolf Riesen,et al.  PUMA: an operating system for massively parallel systems , 1994, 1994 Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences.

[24]  David Kotz,et al.  Low-Level Interfaces for High-Level Parallel I/O , 1995, Input/Output in Parallel and Distributed Computer Systems.

[25]  Carla Schlatter Ellis,et al.  File-Access Characteristics of Parallel Scientific Workloads , 1996, IEEE Trans. Parallel Distributed Syst..

[26]  Randy H. Katz,et al.  Input/output behavior of supercomputing applications , 1991, Proceedings of the 1991 ACM/IEEE Conference on Supercomputing (Supercomputing '91).

[27]  Daniel A. Reed,et al.  Performance of the CM-5 scalable file system , 1994, ICS '94.

[28]  Phillip M. Dickens,et al.  A comparison of the architecture and performance of two parallel file systems , 1989 .

[29]  Thomas H. Cormen,et al.  Early Experiences in Evaluating the Parallel Disk Model with the ViC* Implementation , 1996, Parallel Comput..

[30]  Dror G. Feitelson,et al.  Design and implementation of the Vesta parallel file system , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[31]  Michael L. Best,et al.  CMMD I/O: a parallel Unix I/O , 1993, [1993] Proceedings Seventh International Parallel Processing Symposium.

[32]  Philip J. Hatcher,et al.  Efficient data-parallel files via automatic mode detection , 1996, IOPADS '96.

[33]  David Kotz,et al.  Integrating Theory and Practice in Parallel File Systems , 1993 .