Capsules Encoder and Capsgan for Image Inpainting

Convolutional neural networks (CNN) have solved a lot of tasks in the field of computer vision because of its powerful processing capacity in recent years. Many different kinds of CNN are used to implement in the image restoration tasks and achieved outstanding results. Hinton propose a new network architecture with dynamic routing called Capsule networks which has shown remarkable results for image classification on MNIST data. Its advantage over CNN is that the networks can preserve more information from original image feature map with dynamic routing and compress the image into a vector, which CNN process the image as a scalar. We expand the capsule networks to the task of image restoration for the first time. Our idea is based on the structure of encoder-decoder by replacing convolutional neural networks with capsules networks in encoder aspect. Especially, we add Generative Adversarial Capsule Network (CapsuleGAN) in decoder to constraint parameters for expanding generative space. We evaluate our method on the ImageNet datasets and achieved acceptable results.