Deep Feature-Preserving Based Face Hallucination: Feature Discrimination Versus Pixels Approximation

Face hallucination aims to produce a high-resolution (HR) face image from an input low-resolution (LR) face image, which is of great importance for many practical face applications, such as face recognition and face verification. Since the structure features of face image is complex and sensitive, obtaining a super-resolved face image is more difficult than generic image super-resolution (SR). To address these limitations, we present a novel GAN (Generative adversarial network) based feature-preserving face hallucination approach for very low resolution (\(16 \times 16\) pixels) faces and large scale upsampling (8\(\times \)). Specifically, we design a new residual structure based face generator and adopt two different discriminators - an image discriminator and a feature discriminator, to encourage the model to acquire more realistic face features rather than artifacts. The evaluations based on both PSNR and visual result reveal that the proposed model is superior to the state-of-the-art methods.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Xin Yu,et al.  Hallucinating Very Low-Resolution Unaligned and Noisy Face Images by Transformative Discriminative Autoencoders , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Jian Yang,et al.  FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Xin Yu,et al.  Face Super-Resolution Guided by Facial Component Heatmaps , 2018, ECCV.

[6]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Xin Yu,et al.  Face Hallucination with Tiny Unaligned Images by Transformative Discriminative Neural Networks , 2017, AAAI.

[8]  Aruna Bhat Makeup Invariant Face Recognition using Features from Accelerated Segment Test and Eigen Vectors , 2017, Int. J. Image Graph..

[9]  Georgios Tzimiropoulos,et al.  Super-FAN: Integrated Facial Landmark Localization and Super-Resolution of Real-World Low Resolution Faces in Arbitrary Poses with GANs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[11]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[13]  Xin Yu,et al.  Super-Resolving Very Low-Resolution Face Images with Supplementary Attributes , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Tieniu Tan,et al.  Wavelet-SRNet: A Wavelet-Based CNN for Multi-scale Face Super Resolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[15]  Xin Yu,et al.  Ultra-Resolving Face Images by Discriminative Generative Networks , 2016, ECCV.

[16]  Takeo Kanade,et al.  Hallucinating faces , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[17]  Bernhard Schölkopf,et al.  EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[18]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).