Multicore-enabling the MPJ express messaging library

With the transition to multicore processors almost complete, the parallel processing community is seeking efficient ways to port legacy message passing applications on shared memory and multicore processors. MPJ Express is our reference implementation of Message Passing Interface (MPI)-like bindings for the Java language. Starting with the current release, the MPJ Express software can be configured in two modes: the multicore and the cluster mode. In the multicore mode, parallel Java applications execute on shared memory or multicore processors. In the cluster mode, Java applications parallelized using MPJ Express can be executed on distributed memory platforms like compute clusters and clouds. The multicore device has been implemented using Java threads in order to satisfy two main design goals of portability and performance. We also discuss the challenges of integrating the multicore device in the MPJ Express software. This turned out to be a challenging task because the parallel application executes in a single JVM in the multicore mode. On the contrary in the cluster mode, the parallel user application executes in multiple JVMs. Due to these inherent architectural differences between the two modes, the MPJ Express runtime is modified to ensure correct semantics of the parallel program. Towards the end, we compare performance of MPJ Express (multicore mode) with other C and Java message passing libraries---including mpiJava, MPJ/Ibis, MPICH2, MPJ Express (cluster mode)---on shared memory and multicore processors. We found out that MPJ Express performs signicantly better in the multicore mode than in the cluster mode. Not only this but the MPJ Express software also performs better in comparison to other Java messaging libraries including mpiJava and MPJ/Ibis when used in the multicore mode on shared memory or multicore processors. We also demonstrate effectiveness of the MPJ Express multicore device in Gadget-2, which is a massively parallel astrophysics N-body siimulation code.

[1]  G. P. Nikishkov,et al.  Comparison of C and Java performance in finite element computations , 2003 .

[2]  Aamir Shafi,et al.  Towards efficient shared memory communications in MPJ express , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[3]  Geoffrey C. Fox,et al.  HPJava: Data Parallel Extensions to Java , 1998, Concurr. Pract. Exp..

[4]  Rob van Nieuwpoort,et al.  MPJ/Ibis: A Flexible and Efficient Message Passing Platform for Java , 2005, PVM/MPI.

[5]  V. Springel.  The Cosmological simulation code GADGET-2 , 2005, astro-ph/0505010.

[6]  Geoffrey C. Fox.  Editorial: Java for computational science and engineering – simulation and modeling , 1997 .

[7]  Hans Werner Meuer,et al.  Top500 Supercomputer Sites , 1997 .

[8]  J. Mark Bull,et al.  Benchmarking Java against C and Fortran for scientific applications , 2001, JGI '01.

[9]  Geoffrey C. Fox,et al.  MPIJAVA: An Object-Oriented JAVA Interface to MPI , 1999, IPPS/SPDP Workshops.

[10]  Geoffrey C. Fox,et al.  HPJava: data parallel extensions to Java , 1998 .

[11]  George Bosilca,et al.  Open MPI: Goals, Concept, and Design of a Next Generation MPI Implementation , 2004, PVM/MPI.

[12]  Mark Baker,et al.  MPJ Express Meets Gadget: Towards a Java Code for Cosmological Simulations , 2006, PVM/MPI.

[13]  Message P Forum,et al.  MPI: A Message-Passing Interface Standard , 1994 .

[14]  Samuel P. Midkiff,et al.  From flop to megaflops: Java for technical computing , 1998, TOPL.

[15]  James R. Larus,et al.  Software and the Concurrency Revolution , 2005, ACM Queue.

[16]  Geoffrey C. Fox.  Editorial: Java for computational science and engineering - simulation and modeling , 1997, Concurr. Pract. Exp..

[17]  Mark Baker,et al.  Nested parallelism for multi-core HPC systems using Java , 2009, J. Parallel Distributed Comput..

[18]  William Gropp,et al.  MPICH2: A New Start for MPI Implementations , 2002, PVM/MPI.

[19]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[20]  Geoffrey C. Fox,et al.  mpiJava 1.2: API Specification , 1999 .