Extending the POSIX I/O interface: a parallel file system perspective.

The POSIX interface does not lend itself well to enabling good performance for high-end applications. Extensions are needed in the POSIX I/O interface so that high-concurrency HPC applications running on top of parallel file systems perform well. This paper presents the rationale, design, and evaluation of a reference implementation of a subset of the POSIX I/O interfaces on a widely used parallel file system (PVFS) on clusters. Experimental results on a set of micro-benchmarks confirm that the extensions to the POSIX interface greatly improve scalability and performance.

[1]  Mahadev Satyanarayanan,et al.  Disconnected Operation in the Coda File System , 1999, Mobidata.

[2]  David Kotz,et al.  Disk-directed I/O for MIMD multiprocessors , 1994, OSDI '94.

[3]  Carl Smith,et al.  NFS Version 3: Design and Implementation , 1994, USENIX Summer.

[4]  David Kotz,et al.  The galley parallel file system , 1997, ICS '96.

[5]  Rajeev Thakur,et al.  On implementing MPI-IO portably and with high performance , 1999, IOPADS '99.

[6]  Mahmut T. Kandemir,et al.  On the performance of the POSIX I/O interface to PVFS , 2004, 12th Euromicro Conference on Parallel, Distributed and Network-Based Processing, 2004. Proceedings..

[7]  Rajeev Thakur,et al.  Users guide for ROMIO: A high-performance, portable MPI-IO implementation , 1997 .

[8]  Rajeev Thakur,et al.  UPC-IO: A Parallel I/O API for UPC , 2003 .

[9]  Rajeev Thakur,et al.  Users Guide for ROMIO: A High-Performance , 1997 .

[10]  David Kotz,et al.  Low-Level Interfaces for High-Level Parallel I/O , 1995, Input/Output in Parallel and Distributed Computer Systems.

[11]  Dan Walsh,et al.  Design and implementation of the Sun network filesystem , 1985, USENIX Conference Proceedings.

[12]  Florin Isaila,et al.  Clusterfile: a flexible physical layout parallel file system , 2001, Proceedings 42nd IEEE Symposium on Foundations of Computer Science.

[13]  Andrew A. Chien,et al.  I/O requirements of scientific applications: an evolutionary view , 1996, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing.

[14]  Robert Latham,et al.  A next-generation parallel file system for Linux cluster. , 2004 .

[15]  Robert B. Ross,et al.  PVFS: A Parallel File System for Linux Clusters , 2000, Annual Linux Showcase & Conference.

[16]  Ieee Standards Board System application program interface (API) (C language) , 1990 .

[17]  David A. Pease,et al.  Beyond backup toward storage management , 2003, IBM Syst. J..

[18]  B. Fryxell,et al.  FLASH: An Adaptive Mesh Hydrodynamics Code for Modeling Astrophysical Thermonuclear Flashes , 2000 .