I/O analysis and optimization for an AMR cosmology application

In this paper we investigate the data access patterns and file I/O behaviors of a production cosmology application that uses the adaptive mesh refinement (AMR) technique for its domain decomposition. This application was originally developed using Hierarchical Data Format (HDF version 4) I/O library and since HDF4 does not provide parallel I/O facilities, the global file I/O operations were carried out by one of the allocated processors. When the number of processors becomes large, the I/O performance of this design degrades significantly due to the high communication cost and sequential file access. In this work, we present two additional I/O implementations, using MPI-IO and parallel HDF version 5, and analyze their impacts to the I/O performance for this typical AMR application. Based on the I/O patterns discovered in this application, we also discuss the interaction between user level parallel I/O operations and different parallel file systems and point out the advantages and disadvantages. The performance results presented in this work are obtained from an SGI Origin2000 using XFS, an IBM SP using GPFS, and a Linux cluster using PVFS.

[1]  Greg L. Bryan,et al.  Fluids in the universe: adaptive mesh refinement in cosmology , 1999, Comput. Sci. Eng..

[2]  John Shalf,et al.  Diving deep: data-management and visualization strategies for adaptive mesh refinement simulations , 1999, Comput. Sci. Eng..

[3]  Gil Utard,et al.  MPI-IO on a parallel file system for cluster of workstations , 1999, ICWC 99. IEEE Computer Society International Workshop on Cluster Computing.

[4]  Robert B. Ross,et al.  Using MPI-2: Advanced Features of the Message Passing Interface , 2003, CLUSTER.

[5]  Ralph A. Nelson,et al.  Three-Dimensional Reconstruction Of Tissues And Organs From Sections At The National Center For Supercomputing Applications , 1989, Other Conferences.

[6]  Alok Choudhary,et al.  PASSION Runtime Library for parallel I/O , 1994, Proceedings Scalable Parallel Libraries Conference.

[7]  Wei-keng Liao,et al.  Meta-data Management System for High-Performance Large-Scale Scientific Data Access , 2000, HiPC.

[8]  Rajeev Thakur,et al.  Data sieving and collective I/O in ROMIO , 1998, Proceedings. Frontiers '99. Seventh Symposium on the Frontiers of Massively Parallel Computation.

[9]  P. Colella,et al.  Local adaptive mesh refinement for shock hydrodynamics , 1989 .

[10]  Rajeev Thakur,et al.  An Extended Two-Phase Method for Accessing Sections of Out-of-Core Arrays , 1996, Sci. Program..

[11]  Zhiling Lan,et al.  Dynamic Load Balancing of SAMR Applications on Distributed Systems , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[12]  Rajeev Thakur,et al.  Users guide for ROMIO: A high-performance, portable MPI-IO implementation , 1997 .

[13]  E. Lusk,et al.  An abstract-device interface for implementing portable parallel-I/O interfaces , 1996, Proceedings of 6th Symposium on the Frontiers of Massively Parallel Computation (Frontiers '96).

[14]  Zhiling Lan,et al.  Dynamic load balancing for structured adaptive mesh refinement applications , 2001, International Conference on Parallel Processing, 2001..

[15]  Alok N. Choudhary,et al.  Improved parallel I/O via a two-phase run-time access strategy , 1993, CARN.

[16]  Rakesh Krishnaiyer,et al.  PASSION: Parallel And Scalable Software for Input-Output , 1994 .