Blind single image super-resolution with a mixture of deep networks

Abstract Existing deep neural network based image super-resolution (SR) methods are mostly designed for non-blind cases, where the blur kernel used to generate the low-resolution (LR) images is assumed to be known and fixed. However, this assumption does not hold in many real scenarios. Motivated by the observation that SR of LR images generated by different blur kernels are essentially different but also correlated, we propose a mixture model of deep networks, which is capable of clustering SR tasks of different blur kernels into a set of groups. Each group is composed of correlated SR tasks with similar blur kernels and can be effectively handled by a combination of specific networks in the mixture model. To achieve automatic SR tasks clustering and network selection, we model the blur kernel with a latent variable, which is inferred from the input image by an encoder network. Since the ground-truth of the latent variable is unknown in the training stage, we initialize the encoder network by pre-training it on the blur kernel classification task to avoid trivial solutions. To jointly train the mixture model and the encoder network, we further derive a lower bound of the likelihood function, which circumvents the intractability in direct maximum likelihood estimation. Extensive evaluations are performed on benchmark data sets and validate the effectiveness of the proposed method.