论文信息 - Neural Networks on GPUs : Restricted Boltzmann Machines

Neural Networks on GPUs : Restricted Boltzmann Machines

Despite the popularity and success of neural networks in research, the number of resulting commercial or industrial applications have been limited. A primary cause of this lack of adoption is due to the fact that neural networks are usually implemented as software running on general-purpose processors. In this paper, we investigate how GPUs can be used to take advantage of the inherent parallelism in neural networks to provide a better implementation in terms of performance. We will focus on the Restricted Boltzmann machine, a popular type of neural network. The algorithm is tested on a NVIDIA GTX280 GPU, resulting in a computational speed of 672 million connections-per-second and a speed-up of 66-fold over an optimized C++ program running on a 2.83GHz Intel processor.

D. Ly | Volodymyr Paprotski | D. Yen

[1] David Haussler,et al. Unsupervised learning of distributions on binary vectors using two layer networks , 1991, NIPS 1991.

[2] Matti Tommiska,et al. Efficient digital implementation of the sigmoid function for reprogrammable logic , 2003 .

[3] Indranil Saha,et al. Artiflcial Neural Networks in Hardware: A Survey , 2008 .

[4] Keechul Jung,et al. Neural Network Implementation Using CUDA and OpenMP , 2008, 2008 Digital Image Computing: Techniques and Applications.

[5] Paul Chow,et al. A high-performance FPGA architecture for restricted boltzmann machines , 2009, FPGA '09.

[6] Rajat Raina,et al. Large-scale deep unsupervised learning using graphics processors , 2009, ICML '09.