Object features and face detection performance: Analyses with 3D-rendered synthetic data

This paper is to provide an overview of how object features from images influence face detection performance, and how to select synthetic faces to address specific features. To this end, we investigate the effects of occlusion, scale, viewpoint, background, and noise by using a novel synthetic image generator based on 3DU Face Dataset. To examine the effects of different features, we selected three detectors (Faster RCNN, HR, SSH) as representative of various face detection methodologies. Comparing different configurations of synthetic data on face detection systems, it showed that our synthetic dataset could complement face detectors to become more robust against features in the real world. Our analysis also demonstrated that a variety of data augmentation is necessary to address nuanced differences in performance.

[1]  Xi Zhou,et al.  Data augmentation for face recognition , 2017, Neurocomputing.

[2]  Shiguang Shan,et al.  Real-Time Rotation-Invariant Face Detection with Progressive Calibration Networks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Geoff S. Nitschke,et al.  Improving Deep Learning with Generic Data Augmentation , 2018, 2018 IEEE Symposium Series on Computational Intelligence (SSCI).

[4]  Larry S. Davis,et al.  An Analysis of Scale Invariance in Object Detection - SNIP , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[5]  Luis Perez,et al.  The Effectiveness of Data Augmentation in Image Classification using Deep Learning , 2017, ArXiv.

[6]  Francesc Moreno-Noguer,et al.  GANimation: Anatomically-aware Facial Animation from a Single Image , 2018, ECCV.

[7]  Tal Hassner,et al.  Do We Really Need to Collect Millions of Faces for Effective Face Recognition? , 2016, ECCV.

[8]  Ersin Yumer,et al.  Neural Face Editing with Intrinsic Image Disentangling , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Xu Tang,et al.  PyramidBox: A Context-assisted Single Shot Face Detector , 2018, ECCV.

[10]  Shuo Yang,et al.  WIDER FACE: A Face Detection Benchmark , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Du-Sik Park,et al.  Rotating your face using multi-task deep neural network , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Tal Hassner,et al.  Face-Specific Data Augmentation for Unconstrained Face Recognition , 2019, International Journal of Computer Vision.

[13]  Mingyuan Zhou,et al.  Hybrid sensing face detection and registration for low-light and unconstrained conditions. , 2018, Applied optics.

[14]  Xiangyu Zhu,et al.  High-fidelity Pose and Expression Normalization for face recognition in the wild , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Gang Hua,et al.  Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Sergio Guadarrama,et al.  Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Matthew Turk,et al.  A Morphable Model For The Synthesis Of 3D Faces , 1999, SIGGRAPH.

[18]  Peiyun Hu,et al.  Finding Tiny Faces , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Derek Hoiem,et al.  Diagnosing Error in Object Detectors , 2012, ECCV.

[20]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Shifeng Zhang,et al.  S^3FD: Single Shot Scale-Invariant Face Detector , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[22]  Thomas S. Huang,et al.  Survey of Face Detection on Low-Quality Images , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[23]  Bernhard Egger,et al.  Training Deep Face Recognition Systems with Synthetic Data , 2018, ArXiv.

[24]  Wei Shen,et al.  Learning Residual Images for Face Attribute Manipulation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Stan Z. Li,et al.  Single-Shot Scale-Aware Network for Real-Time Face Detection , 2019, International Journal of Computer Vision.

[26]  Hans-Peter Seidel,et al.  Fast Face Detector Training Using Tailored Views , 2013, 2013 IEEE International Conference on Computer Vision.

[27]  Erik Learned-Miller,et al.  FDDB: A benchmark for face detection in unconstrained settings , 2010 .

[28]  Gang Yu,et al.  Face Attention Network: An Effective Face Detector for the Occluded Faces , 2017, ArXiv.

[29]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Yan Wang,et al.  GenFace: Improving Cyber Security Using Realistic Synthetic Face Generation , 2017, CSCML.

[31]  Ramakant Nevatia,et al.  A multi-scale cascade fully convolutional network face detector , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[32]  Shiguo Lian,et al.  A survey on face data augmentation for the training of deep neural networks , 2019, Neural Computing and Applications.

[33]  Sridha Sridharan,et al.  Using Synthetic Data to Improve Facial Expression Analysis with 3D Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[34]  Shiming Ge,et al.  Detecting Masked Faces in the Wild with LLE-CNNs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Zhe Chen,et al.  Context Refinement for Object Detection , 2018, ECCV.

[36]  Vishal M. Patel,et al.  Pushing the Limits of Unconstrained Face Detection: a Challenge Dataset and Baseline Results , 2018, 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[37]  Guoyong Qiu,et al.  A survey of virtual sample generation technology for face recognition , 2018, Artificial Intelligence Review.

[38]  Larry S. Davis,et al.  SSH: Single Stage Headless Face Detector , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[39]  Stefanos Zafeiriou,et al.  A survey on face detection in the wild: Past, present and future , 2015, Comput. Vis. Image Underst..

[40]  Xiaolin Hu,et al.  Scale-Aware Face Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Tal Hassner,et al.  Effective face frontalization in unconstrained images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).