Regularization Learning for Image Recognition

In order to reduce overfitting for the image recognition application, this paper proposes a novel regularization learning algorithm for deep learning. Above all, we propose a novel probabilistic representation for explaining the architecture of Deep Neural Networks (DNNs), which demonstrates that the hidden layers close to the input formulate prior distributions, thus DNNs have an explicit regularization, namely the prior distributions. Furthermore, we show that the backpropagation learning algorithm is the reason for overfitting because it cannot guarantee precisely learning the prior distribution. Based on the proposed theoretical explanation for deep learning, we propose a novel regularization learning algorithm for DNNs. In contrast to most existing regularization methods reducing overfitting by decreasing the training complexity of DNNs, the proposed method reduces overfitting through training the corresponding prior distribution in a more efficient way, thereby deriving a more powerful regularization. Simulations demonstrate the proposed probabilistic representation on a synthetic dataset and validate the proposed regularization on the CIFAR-10 dataset.