A library for parallel IO and data management has been developed for multi-physics simulations. The goal of the library is to provide sustainable, interoperable, efficient, scalable, and convenient tools for parallel IO and data management for high-level data structures in numerical simulations, and to provide tools for the connection between applications. The high-level data structures include one- and multi-dimensional arrays, structured meshes, unstructured meshes, the meshes generated through (block-based, patch-based, and cell-based) adaptive mesh refinement, variables associated with these meshes, and data defined on particles in particle simulations. The IO mechanism can be collective and non-collective. The library is typically used for restarting files, visualization files, and files connecting different applications. The library is based on MPI-IO. Compared with the IO performance of MPI-IO, the overhead to write the explicit users' data structures are less than five percent. To further improve IO performance, in addition to the bookkeeping data, the library could buffer problem-size data before calling MPI-IO while keeping users' explicit high-level data structures. The buffering mechanism improves IO performance by a factor 10 to 20 in multi-physics simulations involving AMR and unstructured meshes.
[1]
Mark C. Miller,et al.
Enabling Interoperation of High Performance, Scientific Computing Applications: Modeling Scientific Data with the Sets & Fields (SAF) Modeling System
,
2001,
International Conference on Computational Science.
[2]
Russ Rew,et al.
NetCDF: an interface for scientific data access
,
1990,
IEEE Computer Graphics and Applications.
[3]
Paul F. Dubois,et al.
Software for Portable Scientific Data Management
,
1993
.
[4]
Thomas Hauser.
Parallel I/O for the CGNS System
,
2004
.