Towards Efficient Support for Parallel I/O in Java HPC

Modern HPC applications put forward significant I/O requirements. To deal with them, MPI provides the MPI-IO API for parallel file access. ROMIO library implements MPI-IO and provides efficient support for parallel I/O in C and Fortran based applications. On the other hand, Java based MPI-like libraries such as MPJ Express and F-MPJ have emerged but they lack parallel I/O support. Little research has been done to provide Java based ROMIO-like libraries due to the non-availability of MPI-IO-like API for the Java language. In this paper, we take the first step towards the development of parallel I/O API in Java by evaluating the newly introduced Java NIO API versus the legacy Java I/O API. We propose two simple approaches for performing parallel file I/O using NIO and evaluate them on two different computational platforms. The implementation of proposed approaches exploits the view buffers concept of NIO API to perform efficient array based file I/O operations from multiple processes. We report encouraging speedups and suggest that design of a parallel I/O API in Java should be based on the NIO API.

[1]  Juan Touriño,et al.  F-MPJ: scalable Java message-passing communications on parallel systems , 2012, The Journal of Supercomputing.

[2]  Rajeev Thakur,et al.  Users guide for ROMIO: A high-performance, portable MPI-IO implementation , 1997 .

[3]  High performance Java input/output for heterogeneous distributed computing , 2005, 10th IEEE Symposium on Computers and Communications (ISCC'05).

[4]  Rajeev Thakur,et al.  High‐performance file I/O in Java: Existing approaches and bulk I/O extensions , 2001, Concurr. Comput. Pract. Exp..

[5]  Rob van Nieuwpoort,et al.  MPJ/Ibis: A Flexible and Efficient Message Passing Platform for Java , 2005, PVM/MPI.

[6]  Geoffrey C. Fox,et al.  mpiJava 1.2: API Specification , 1999 .

[7]  M. Fukuda,et al.  A Java Implementation of MPI-I/O-Oriented Random Access File Class in AgentTeamwork Grid Computing Middleware , 2007, 2007 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing.

[8]  Alan Kaminsky Parallel Java: A Unified API for Shared Memory and Cluster Parallel Programming in 100% Java , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[9]  George Bosilca,et al.  Open MPI: Goals, Concept, and Design of a Next Generation MPI Implementation , 2004, PVM/MPI.

[10]  Robert B. Ross,et al.  Using MPI-2: Advanced Features of the Message Passing Interface , 2003, CLUSTER.

[11]  Yuichi Tsujita,et al.  MPI-I/O operations to a remote computer using Java , 2005, 11th International Conference on Parallel and Distributed Systems (ICPADS'05).

[12]  Dan Bonachea Bulk file I/O extensions to Java , 2000, JAVA '00.

[13]  Rajeev Thakur,et al.  I/O in Parallel Applications: the Weakest Link , 1998, Int. J. High Perform. Comput. Appl..

[14]  Rajeev Thakur,et al.  An evaluation of Java's I/O capabilities for high-performance computing , 2000, JAVA '00.

[15]  Jesús Carretero,et al.  The Design of the Expand Parallel File System , 2003, Int. J. High Perform. Comput. Appl..