UP-CNN: Un-pooling augmented convolutional neural network

Abstract Convolutional neural network (CNN) has shown remarkable performance in various visual recognition tasks. Most of existing CNN is a purely bottom-up and feed-forward architecture, we argue that it fails to consider the interaction between low-level fine details and high-level semantic information. In this paper, a novel “Un-Pooling augmented Convolutional Neural Network” (UP-CNN) is proposed to boost the discriminative capability of the CNN with the following three distinctive properties: (1) UP-CNN is a deeper network, which is comprised of a bottom-up, a top-down and then a bottom-up sub-networks, associating to different level information that jointly improves its discriminative capability. (2) With the mixture of pooling and un-pooling layers, UP-CNN easily allows the interaction across convolutional layers with the same size from different sub-networks. This architecture effectively depresses the attenuation of important information including both activations in the forward process and gradient information in the back-propagation process. (3) UP-CNN employs the ratio un-pooling operation to reconstruct activations of the original size in the top-down sub-network, where the spatial information that is lost during pooling can be preserved within a receptive field. The experiments on four benchmark datasets (including the CIFAR-10, CIFAR-100, MNIST and SVHN datasets) well demonstrate that the proposed UP-CNN architecture considerably outperforms other state-of-the-art methods.