Use of a new I / O stack for extreme-scale systems in scientific applications

Exascale I/O initiatives will require new and fully integrated I/O models which are capable of providing straightforward functionality, fault tolerance and efficiency. One solution is the development of a transactional object storage I/O stack and an extended version of HDF5 capable of managing this next generation I/O stack. This new HDF5 implementation adds support for end-to-end data integrity and security, mapping objects, index building, maintenance and query, and analysis shipping. This research highlights challenges and issues when porting common scientific application codes to use a transactional I/O stack.

[1]  P. Nowoczynski,et al.  Zest Checkpoint storage system for large supercomputers , 2008, 2008 3rd Petascale Data Storage Workshop.

[2]  Carlos Maltzahn,et al.  Efficient transactions for parallel data movement , 2013, PDSW@SC.

[3]  David E. Keyes,et al.  Exaflop/s: The why and the how , 2011 .

[4]  Sorin Faibish,et al.  Jitter-free co-processing on a prototype exascale storage stack , 2012, 012 IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST).

[5]  Rudolf Eigenmann,et al.  Parallel I/O Library (PIO) , 2011, Encyclopedia of Parallel Computing.

[6]  Jesús Carretero,et al.  Making the case for reforming the I/O software stack of extreme-scale systems , 2017, Adv. Eng. Softw..

[7]  Kshitij Mehta,et al.  A Plugin for HDF5 Using PLFS for Improved I/O Performance and Semantic Analysis , 2012, 2012 SC Companion: High Performance Computing, Networking Storage and Analysis.