EDFace-Celeb-1 M: Benchmarking Face Hallucination With a Million-Scale Dataset

Recent deep face hallucination methods show stunning performance in super-resolving severely degraded facial images, even surpassing human ability. However, these algorithms are mainly evaluated on non-public synthetic datasets. It is thus unclear how these algorithms perform on public face hallucination datasets. Meanwhile, most of the existing datasets do not well consider the distribution of races, which makes face hallucination methods trained on these datasets biased toward some specific races. To address the above two problems, in this paper, we build a public Ethnically Diverse Face dataset, EDFace-Celeb-1 M, and design a benchmark task for face hallucination. Our dataset includes 1.7 million photos that cover different countries, with relatively balanced race composition. To the best of our knowledge, it is the largest-scale and publicly available face hallucination dataset in the wild. Associated with this dataset, this paper also contributes various evaluation protocols and provides comprehensive analysis to benchmark the existing state-of-the-art methods. The benchmark evaluations demonstrate the performance and limitations of state-of-the-art algorithms. https://github.com/HDCVLab/EDFace-Celeb-1M.

[1]  Zixiang Xiong,et al.  Dual-Path Deep Fusion Network for Face Image Hallucination , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[2]  Nick Barnes,et al.  Densely Residual Laplacian Super-Resolution , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Tal Hassner,et al.  img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Kimmo Kärkkäinen,et al.  FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age , 2019, ArXiv.

[5]  Steven C. H. Hoi,et al.  Deep Learning for Image Super-Resolution: A Survey , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Xiaochun Cao,et al.  Correction to: Single Image Super-Resolution via a Holistic Attention Network , 2020, ECCV.

[7]  Lei Zhang,et al.  Blind Face Restoration via Deep Multi-scale Component Dictionaries , 2020, ECCV.

[8]  W. Gao,et al.  HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment , 2020, ACM Multimedia.

[9]  Jie Zhou,et al.  Deep Face Super-Resolution With Iterative Collaboration Between Attentive Recovery and Landmark Estimation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Changhui Hu,et al.  Copy and Paste GAN: Face Hallucination From Shaded Thumbnails , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Dae-Shik Kim,et al.  Progressive Face Super-Resolution via Attention to Facial Landmark , 2019, BMVC.

[12]  Ming Zeng,et al.  Face Parsing With RoI Tanh-Warping , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Shu-Tao Xia,et al.  Second-Order Attention Network for Single Image Super-Resolution , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Wei Wu,et al.  Feedback Network for Image Super-Resolution , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Wei Liu,et al.  Super-Identity Convolutional Neural Network for Face Hallucination , 2018, ECCV.

[18]  Xin Yu,et al.  Face Super-Resolution Guided by Facial Component Heatmaps , 2018, ECCV.

[19]  Yu Qiao,et al.  ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks , 2018, ECCV Workshops.

[20]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[21]  Xin Yu,et al.  Super-Resolving Very Low-Resolution Face Images with Supplementary Attributes , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Gregory Shakhnarovich,et al.  Deep Back-Projection Networks for Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Yun Fu,et al.  Residual Dense Network for Image Super-Resolution , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Xing Ji,et al.  CosFace: Large Margin Cosine Loss for Deep Face Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[25]  Jian Yang,et al.  FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Omkar M. Parkhi,et al.  VGGFace2: A Dataset for Recognising Faces across Pose and Age , 2017, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[27]  Ming-Hsuan Yang,et al.  Hallucinating Compressed Face Images , 2018, International Journal of Computer Vision.

[28]  Tieniu Tan,et al.  Wavelet-SRNet: A Wavelet-Based CNN for Multi-scale Face Super Resolution , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Jiawei Zhang,et al.  Learning to Hallucinate Face Images via Component Generation and Enhancement , 2017, IJCAI.

[30]  Kyoung Mu Lee,et al.  Enhanced Deep Residual Networks for Single Image Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[31]  Liang Lin,et al.  Attention-Aware Face Hallucination via Deep Reinforcement Learning , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Narendra Ahuja,et al.  Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Georgios Tzimiropoulos,et al.  How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks) , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[34]  Bernhard Schölkopf,et al.  EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[35]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Xin Yu,et al.  Ultra-Resolving Face Images by Discriminative Generative Networks , 2016, ECCV.

[37]  Xiaoou Tang,et al.  Accelerating the Super-Resolution Convolutional Neural Network , 2016, ECCV.

[38]  Xiaoou Tang,et al.  Deep Cascaded Bi-Network for Face Hallucination , 2016, ECCV.

[39]  Ira Kemelmacher-Shlizerman,et al.  The MegaFace Benchmark: 1 Million Faces for Recognition at Scale , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Xiangyu Zhu,et al.  Face Alignment in Full Pose Range: A 3D Total Solution , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Kyoung Mu Lee,et al.  Deeply-Recursive Convolutional Network for Image Super-Resolution , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Gustavo K. Rohde,et al.  Transport-based single frame super resolution of very low resolution face images , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Ming-Hsuan Yang,et al.  Multi-objective convolutional learning for face labeling , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Yuning Jiang,et al.  Learning Face Hallucination in the Wild , 2015, AAAI.

[47]  Xiaogang Wang,et al.  Deep Learning Face Attributes in the Wild , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[48]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[49]  Shengcai Liao,et al.  Learning Face Representation from Scratch , 2014, ArXiv.

[50]  Xiaoou Tang,et al.  Learning a Deep Convolutional Network for Image Super-Resolution , 2014, ECCV.

[51]  Ruimin Hu,et al.  Face Super-Resolution via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning , 2014, IEEE Transactions on Image Processing.

[52]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  Xiaogang Wang,et al.  Deep Convolutional Network Cascade for Facial Point Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Chih-Yuan Yang,et al.  Structured Face Hallucination , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Xuelong Li,et al.  A Comprehensive Survey to Face Hallucination , 2013, International Journal of Computer Vision.

[56]  Ce Liu,et al.  A Bayesian Approach to Alignment-Based Image Hallucination , 2012, ECCV.

[57]  Maoguo Gong,et al.  Position-Patch Based Face Hallucination Using Convex Optimization , 2011, IEEE Signal Processing Letters.

[58]  Junping Zhang,et al.  Super-resolution of human face image using canonical correlation analysis , 2010, Pattern Recognit..

[59]  Chun Qi,et al.  Hallucinating face by position-patch , 2010, Pattern Recognit..

[60]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[61]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[62]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[63]  Shaogang Gong,et al.  Generalized Face Super-Resolution , 2008, IEEE Transactions on Image Processing.

[64]  Rama Chellappa,et al.  Super-Resolution of Face Images Using Kernel PCA-Based Prior , 2007, IEEE Transactions on Multimedia.

[65]  Harry Shum,et al.  Face Hallucination: Theory and Practice , 2007, International Journal of Computer Vision.

[66]  Xiaogang Wang,et al.  Hallucinating face by eigentransformation , 2005, IEEE Trans. Syst. Man Cybern. Part C.