High performance Java input/output for heterogeneous distributed computing

Currently there is a growing interest in using Java for high performance computing. Java has many advantages for high performance computing: it is based on a high-level and object-oriented programming model with support for multithreading and distributed computing. Furthermore, Java 's virtual machine allows applications to run on multiple heterogeneous platforms. A major problem with the use of Java for high performance computing is the I/O. This problem has been solved traditionally in clusters using parallel file systems and parallel I/O libraries, however there is a lack of parallel file systems for Java applications. In this paper, we present a Java parallel I/O library called jExpand. It provides high performance I/O by using several NFS servers in parallel, as NFS can be found in multiple platforms (Linux, Solaris, Windows 2000, etc), we provide a universal parallel file system that can be used everywhere. jExpand requires no changes in the NFS server as it uses RPC operations to provide parallel access to the same file. The paper describes the design, implementation and evaluation of jExpand.

[1]  Raj Srinivasan,et al.  RPC: Remote Procedure Call Protocol Specification Version 2 , 1995, RFC.

[2]  Robert B. Ross,et al.  PVFS: A Parallel File System for Linux Clusters , 2000, Annual Linux Showcase & Conference.

[3]  Jesús Carretero,et al.  The Design of the Expand Parallel File System , 2003, Int. J. High Perform. Comput. Appl..

[4]  Jesús Carretero,et al.  Performance Increase Mechanisms for Parallel and Distributed File Systems , 1997, Parallel Comput..

[5]  Amin Vahdat,et al.  Interposed request routing for scalable network storage , 2000, TOCS.

[6]  Rajeev Thakur,et al.  High‐performance file I/O in Java: Existing approaches and bulk I/O extensions , 2001, Concurr. Comput. Pract. Exp..

[7]  Gene H. Kim,et al.  Bigfoot-NFS : A Parallel File-Striping NFS Server ( Extended Abstract ) , 1994 .

[8]  Rajeev Thakur,et al.  An evaluation of Java's I/O capabilities for high-performance computing , 2000, JAVA '00.

[9]  Jim Zelenka,et al.  The Scotch parallel storage systems , 1995, Digest of Papers. COMPCON'95. Technologies for the Information Superhighway.

[10]  David Kotz,et al.  Dynamic file-access characteristics of a production parallel scientific workload , 1994, Proceedings of Supercomputing '94.

[11]  Michael Stumm,et al.  Hfs: a flexible file system for shared-memory multiprocessors , 1994 .

[12]  Andrew A. Chien,et al.  PPFS: a high performance portable parallel file system , 1995, ICS '95.

[13]  Steven A. Moyer,et al.  PIOUS: a scalable parallel I/O system for distributed computing environments , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[14]  Dror G. Feitelson,et al.  Overview of the Vesta parallel file system , 1993, CARN.

[15]  Jesús Carretero,et al.  An Implementation of MPI-IO on Expand: A Parallel File System Based on NFS Servers , 2002, PVM/MPI.

[16]  Erich Schikuta,et al.  Meta-ViPIOS: Harness Distributed I/O Resources with ViPIOS , 2000, Computación y Sistemas.

[17]  David Kotz,et al.  The galley parallel file system , 1997, ICS '96.

[18]  David E. Culler,et al.  Jaguar: enabling efficient communication and I/O in Java , 2000 .

[19]  Ron Oldfield,et al.  Armada: a parallel file system for computational grids , 2001, Proceedings First IEEE/ACM International Symposium on Cluster Computing and the Grid.

[20]  Dan Walsh,et al.  Design and implementation of the Sun network filesystem , 1985, USENIX Conference Proceedings.