The Optimization of Parallel DBN Based on Spark

Deep Belief Network (DBN) is widely used for modelling and analysis of all kinds of actual problems. However, it’s easy to have a computational bottleneck problem when training DBN in a single computational node. And traditional parallel full-batch gradient descent exists the problem that the speed of convergence is slow when we use it to train DBN. To solve this problem, the article proposes a parallel mini-batch gradient descent algorithm based on Spark and uses it to train DBN. The experiment shows the method is faster than parallel full-batch gradient and the convergence result is better when batch size is relatively small. We use the method to train the DBN, and apply it to text classification. We also discuss how the size of batch impacts on the weights of network. The experiments show that it can improve the precision and recall of text classification compared with SVM when batch size is small.