Exploiting Global Input Output Access Pattern Classification

Parallel input/output systems attempt to alleviate the performance bottleneck that affects many input/output intensive applications. In such systems, an understanding of the application access pattern, especially how requests from multiple processors for different file regions are logically related, is important for optimizing file system performance. We propose a method for automatically classifying these global access patterns and using these global classifications to select and tune file system policies to improve input/output performance. We demonstrate this approach on benchmarks and scientific applications using global classification to automatically select appropriate underlying Intel PFS input/output modes and server buffering strategies.

[1]  Steven A. Cuccaro,et al.  Quantum chemical reaction dynamics on a highly parallel supercomputer , 1991 .

[2]  W. Gropp,et al.  The Scalable I/O Initiative , 1995 .

[3]  David Kotz,et al.  Disk-directed I/O for MIMD multiprocessors , 1994, OSDI '94.

[4]  Andrew S. Grimshaw,et al.  ELFS: object-oriented extensible file systems , 1991, [1991] Proceedings of the First International Conference on Parallel and Distributed Information Systems.

[5]  T.M. Madhyastha,et al.  Intelligent, adaptive file system policy selection , 1996, Proceedings of 6th Symposium on the Frontiers of Massively Parallel Computation (Frontiers '96).

[6]  Jim Zelenka,et al.  Informed prefetching and caching , 1995, SOSP.

[7]  D.A. Reed,et al.  Input/Output Characteristics of Scalable Parallel Applications , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[8]  Stanley B. Zdonik,et al.  Fido: A Cache That Learns to Fetch , 1991, VLDB.

[9]  Carla Schlatter Ellis,et al.  Prefetching in File Systems for MIMD Multiprocessors , 1990, IEEE Trans. Parallel Distributed Syst..

[10]  George Em Karniadakis,et al.  Unstructured spectral element methods for simulation of turbulent flows , 1995 .

[11]  Andrew A. Chien,et al.  PPFS: a high performance portable parallel file system , 1995, ICS '95.

[12]  Aron Kuppermann,et al.  The quantitative prediction and lifetime of a pronounced reactive scattering resonance , 1995 .

[13]  Eugene Charniak,et al.  Statistical language learning , 1997 .

[14]  Geoffrey E. Hinton Connectionist Learning Procedures , 1989, Artif. Intell..

[15]  Michael Stumm,et al.  HFS: A Flexible File System for large-scale Multiprocessors , 1993 .

[16]  Dror G. Feitelson,et al.  Mpi-io: a parallel file i/o interface for mpi , 1995 .

[17]  Andrew A. Chien,et al.  I/O requirements of scientific applications: an evolutionary view , 1996, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing.

[18]  Daniel A. Reed,et al.  Automatic classification of input/output access patterns , 1997 .

[19]  Carla Schlatter Ellis,et al.  Practical prefetching techniques for parallel file systems , 1991, [1991] Proceedings of the First International Conference on Parallel and Distributed Information Systems.

[20]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[21]  K. Korner,et al.  Intelligent caching for remote file service , 1990, Proceedings.,10th International Conference on Distributed Computing Systems.