Deep network for image super-resolution with a dictionary learning layer

The aim of single image super-resolution (SR) is to generate a high-resolution (HR) image from a low-resolution (LR) observable image. In this paper, we address this task by integrating sparse coding and dictionary learning schemes into an end-to-end deep architecture. More specifically, we propose a new non-linear dictionary learning layer composed of a finite number of recurrent units to solve the sparse codes and also to yield the relevant gradients to update the dictionary. In addition, we present a new deep network architecture using the proposed non-linear layers, where two separate parallel dictionaries are adopted to represent the LR and HR images respectively. The whole network is optimized by back propagation, constraining not only reconstruction errors between the restored and the ground truth HR images but also between the sparse codes of the LR and HR image pairs. Various datasets are used to evaluate the performance of the proposed approach and it is shown to outperform many state-of-the-art single image super-resolution algorithms.