Two-Stage Transfer Learning of End-to-End Convolutional Neural Networks for Webpage Saliency Prediction

With the great success of convolutional neural networks (CNN) achieved on various computer vision tasks in recent years, CNN has also been applied in natural image saliency prediction. As a specific visual stimuli, webpages exhibit evident similarities whereas also significant differences from natural image. Consequently, the learned CNN for natural image saliency prediction cannot be directly used to predict webpage saliency. Only a few researches on webpage saliency prediction have been developed till now. In this paper, we propose a simple yet effective scheme of two-stage transfer learning of end-to-end CNN to predict the webpage saliency. In the first stage, the output layer of two typical CNN architectures with instances of AlexNet and VGGNet are reconstructed, and the parameters between the fully connected layers are relearned from a large natural image database for image saliency prediction. In the second stage, the parameters between the fully connected layers are relearned from a scarce webpage database for webpage saliency prediction. In fact, the two-stage transfer learning can be regarded as a task transfer in the first stage and a domain transfer in the second stage, respectively. The experimental results indicate that the proposed two-stage transfer learning of end-to-end CNN can obtain a substantial performance improvement for webpage saliency prediction.