Robust Face Super-Resolution via Locality-Constrained Low-Rank Representation

Learning-based face super-resolution relies on obtaining accurate a priori knowledge from the training data. Representation-based approaches (e.g., sparse representation-based and neighbor embedding-based schemes) decompose the input images using sophisticated regularization techniques. They give reasonably good reconstruction performance. However, in real application scenarios, the input images are often noisy, blurry, or suffer from other unknown degradations. Traditional face super-resolution techniques treat image noise at the pixel level without considering the underlying image structures. In order to rectify this shortcoming, we propose in this paper a unified framework for representation-based face super-resolution by introducing a locality-constrained low-rank representation (LLR) scheme to reveal the intrinsic structures of input images. The low-rank representation part of LLR clusters an input image into the most accurate subspace from a global dictionary of atoms, while the locality constraint enables recovery of local manifold structures from local patches. In addition, low-rank, sparsity, locality, accuracy, and robustness of the representation coefficients are exploited in LLR via regularization. Experiments on the FEI, CMU face database, and real surveillance scenario show that LLR outperforms the state-of-the-art face super-resolution algorithms (e.g., convolutional neural network-based deep learning) both objectively and subjectively.