Performance Evaluation of OpenFOAM* with MPI-3 RMA Routines on Intel® Xeon® Processors and Intel® Xeon Phi™ Coprocessors

OpenFOAM is a software package for solving partial differential equations and is very popular for computational fluid dynamics in the automotive segment. In this work, we describe our evaluation of the performance of OpenFOAM with MPI-3 Remote Memory Access (RMA) one-sided communication on the Intel® Xeon Phi" coprocessor. Currently, OpenFOAM computes on a mesh that is decomposed among different MPI ranks, and it requires a high amount of communication between the neighboring ranks. MPI-3 offers RMA through a new API that decouples communication and synchronization. The aim is to achieve better performance with MPI-3 RMA routines as compared to the current two-sided asynchronous communication routines in OpenFOAM. We also showcase the challenges overcome in order to facilitate the different MPI-3 RMA routines in OpenFOAM. This discussion aims at analyzing the potential of MPI-3 RMA in OpenFOAM and benchmarking the performance on both the processor and the coprocessor. Our work also demonstrates that MPI-3 RMA in OpenFOAM can run in symmetric mode consisting of the Intel® Xeon® E5-2697v3 processor and the Intel® Xeon Phi™ 7120P coprocessor.