Design space exploration of GPU Accelerated cluster systems for optimal data transfer using PCIe bus

Use of accelerators such as GPUs is increasing, but efficient use of GPUs requires making good design choices. Such design choices include type of memory allocation and overlapping concurrency of data transfer with parallel computation. Performance varies with the application, hardware version such as generation of GPU, and software version including programming language drivers. This large number of design decisions makes it nearly impossible to obtain the optimal performance point by directly porting any application. This emphasizes the need for high level design decision guidelines for GPU accelerated cluster systems, applicable to a broad class of applications rather than any specific application. This paper proposes novel design guidelines for GPU accelerated cluster systems, to optimize the data transfer from host (CPU) to device (GPU) using the PCIe bus. In particular, we consider design choices offered by NVIDIA GPUs. Our main contribution is to build design guidelines that are applicable to a broad class of applications. We design 27 different versions of the same micro benchmark, where the design choices made by each version is unique. We observe that a speedup of 2.6x can be obtained just by making good design choices.