Optimized deep belief networks on CUDA GPUs

A deep belief network (DBN) is an important branch of deep learning models and has been successfully applied in many machine learning and pattern recognition fields such as computer vision and speech recognition. However, the training of billions of parameters in DBN is computationally challenging for modern central processing units (CPUs). Many studies have reported the efficient implementations of the pre-training process of DBNs for graphics processing units (GPUs), but few studies have mentioned the fine-tuning process of DBNs. In this paper, we describe an efficient DBN implementation on the GPU, including the pre-training and fine-tuning processes. Experimental results show that our proposed method on the GPU (NVIDIA Tesla K40c) achieves up to 22 speedups on the pre-training process and 33 speedups on the fine-tuning processes compared with conventional CPU (Intel Core i7-4790K) implementations. Moreover, the performance of our algorithm is superior to that of the OpenBLAS library on the CPU and the CUBLAS library on the GPU.