Performance Evaluation and Optimization of HBM-Enabled GPU for Data-Intensive Applications

Graphics processing units (GPUs) are widely used to accelerate data-intensive applications. To improve the performance of data-intensive applications, higher GPU memory bandwidth is desirable. Traditional graphics double data rate memories achieve higher bandwidth by increasing frequency, which leads to excessive power consumption. Recently, a new memory technology called high-bandwidth memory (HBM) based on 3-D die-stacking technology has been used in the latest generation of GPUs, which can provide both high bandwidth and low power consumption with in-package-stacked DRAM memory. However, the capacity of integrated in-package-stacked memory is limited (e.g., only 4 GB for the state-of-the-art HBM-enabled GPU, AMD Radeon Fury X). In this paper, we implement two representative data-intensive applications, convolutional neural network (CNN) and breadth-first search (BFS) on an HBM-enabled GPU to evaluate the improvement brought by the adoption of the HBM, and to investigate techniques to fully unleash the benefits of such HBM-enabled GPU. Based on the evaluation results, we first propose a software pipeline to alleviate the capacity limitation of the HBM for CNN. We then design two programming techniques to improve the utilization of memory bandwidth for BFS application. Experimental results demonstrate that our pipelined CNN training achieves a <inline-formula> <tex-math notation="LaTeX">$1.63\times $ </tex-math></inline-formula> speedup on an HBM-enabled GPU compared with the best high-performance GPU in market, and the two optimization techniques for the BFS algorithm make it at most <inline-formula> <tex-math notation="LaTeX">$24.5\times $ </tex-math></inline-formula> (<inline-formula> <tex-math notation="LaTeX">$9.8\times $ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$2.5\times $ </tex-math></inline-formula> for each technique, respectively) faster than conventional implementations.