Workload Characterization of Input/Output Intensive Parallel Applications

The broadening disparity in the performance of input/output (I/O) devices and the performance of processors and communication links on parallel systems is a major obstacle to achieving high performance for a wide range of parallel applications. I/O hardware and file system parallelism are the keys to bridging this performance gap. A prerequisite to the development of efficient parallel file systems is detailed characterization of the I/O demands of parallel applications. In this paper, we present a comparative study of the I/O access patterns commonly found in I/O intensive parallel applications. Using the Pablo performance analysis environment and its I/O extensions we captured application I/O access patterns and analyzed their interactions with current parallel I/O systems. This analysis has proven instrumental in guiding the development of new application programming interfaces (APIs) for parallel file systems and in developing effective file system policies that can adaptively respond to complex application I/O requirements.

[1]  David Kotz,et al.  Performance of the Galley Parallel File System , 1996 .

[2]  利久 亀井,et al.  California Institute of Technology , 1958, Nature.

[3]  Evgenia Smirni,et al.  I/O, performance analysis, and performance data immersion , 1996, Proceedings of MASCOTS '96 - 4th International Workshop on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[4]  Steven A. Cuccaro,et al.  Quantum chemical reaction dynamics on a highly parallel supercomputer , 1991 .

[5]  Steven A. Moyer,et al.  PIOUS: a scalable parallel I/O system for distributed computing environments , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[6]  W. Gropp,et al.  The Scalable I/O Initiative , 1995 .

[7]  George C. Polyzos,et al.  Dynamic I/O characterization of I/O intensive scientific applications , 1994, Proceedings of Supercomputing '94.

[8]  T.M. Madhyastha,et al.  Intelligent, adaptive file system policy selection , 1996, Proceedings of 6th Symposium on the Frontiers of Massively Parallel Computation (Frontiers '96).

[9]  C. Winstead,et al.  Parallel computation of electron molecule collisions , 1995 .

[10]  Peter F. Corbett,et al.  Proposal for a Common Parallel File System Programming Interface 1.0 , 1996, WWW 1996.

[11]  D.A. Reed,et al.  Input/Output Characteristics of Scalable Parallel Applications , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[12]  Rajeev Thakur,et al.  E cient Compilation of Out-of-core Data Parallel Programs , 1994 .

[13]  Andrew A. Chien,et al.  Input/Output Characteristics of Scalable Parallel Applications , 1995, SC.

[14]  Sivan Toledo,et al.  The design and implementation of SOLAR, a portable library for scalable out-of-core linear algebra computations , 1996, IOPADS '96.

[15]  Dror G. Feitelson,et al.  Mpi-io: a parallel file i/o interface for mpi , 1995 .

[16]  David Kotz,et al.  Dynamic file-access characteristics of a production parallel scientific workload , 1994, Proceedings of Supercomputing '94.

[17]  Dror G. Feitelson,et al.  Overview of the Vesta parallel file system , 1993, CARN.

[18]  Joel H. Saltz,et al.  Jovian: a framework for optimizing parallel I/O , 1994, Proceedings Scalable Parallel Libraries Conference.

[19]  David Kotz,et al.  The galley parallel file system , 1996, ICS '96.

[20]  Andrew A. Chien,et al.  I/O requirements of scientific applications: an evolutionary view , 1996, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing.

[21]  D.A. Reed,et al.  Scalable performance analysis: the Pablo performance analysis environment , 1993, Proceedings of Scalable Parallel Libraries Conference.

[22]  Joel H. Saltz,et al.  Framework for optimizing parallel I/O , 1995 .

[23]  George Em Karniadakis,et al.  Unstructured spectral element methods for simulation of turbulent flows , 1995 .

[24]  Carla Schlatter Ellis,et al.  File-Access Characteristics of Parallel Scientific Workloads , 1996, IEEE Trans. Parallel Distributed Syst..

[25]  Randy H. Katz,et al.  Input/output behavior of supercomputing applications , 1991, Proceedings of the 1991 ACM/IEEE Conference on Supercomputing (Supercomputing '91).

[26]  Ian T. Foster,et al.  ChemIo: High Performance Parallel I/o for Computational Chemistry Applications , 1998, Int. J. High Perform. Comput. Appl..

[27]  Carla Schlatter Ellis,et al.  Characterizing parallel file-access patterns on a large-scale multiprocessor , 1995, IPPS.

[28]  Marianne Winslett,et al.  Server-Directed Collective I/O in Panda , 1995, Proceedings of the IEEE/ACM SC95 Conference.