Lightweight and Effective Facial Landmark Detection using Adversarial Learning with Face Geometric Map Generative Network

Facial landmark detection plays an important role in face analysis tasks. Moreover, it is used as a prerequisite in many facial related applications, the simplicity, as well as effectiveness, is essential in the facial landmark detection. In this paper, we propose an effective facial landmark detection network and an associated learning framework with the geometric prior-generative adversarial network. The geometric prior-generative adversarial network consists of one generator and two discriminators. The generator consists of an encoder and two decoders. The encoder predicts facial landmark points. The decoders generate a facial inner and contour geometric map from predicted landmark points. Generating face geometric maps from predicted landmark points helps the predicted landmark points to represent the face geometric information, including shape and configuration. The discriminators determine that the given geometric maps are generated from actual landmark points or estimated landmark points. Our proposed network is end-to-end trainable, and only the encoder part is used simply as the facial landmark detector in the testing stage. To verify the effectiveness of the proposed method, we have conducted comprehensive experiments with benchmark data sets. The results have shown that the proposed method achieves comparable performances over recently proposed facial landmark detection methods with a simple and effective facial landmark detection network.

[1]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  George Trigeorgis,et al.  Mnemonic Descent Method: A Recurrent Process Applied for End-to-End Face Alignment , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Jian Sun,et al.  Face Alignment Via Component-Based Discriminative Search , 2008, ECCV.

[4]  Hanjiang Lai,et al.  Deep Recurrent Regression for Facial Landmark Detection , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Jian Sun,et al.  Face Alignment at 3000 FPS via Regressing Local Binary Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  David J. Kriegman,et al.  Localizing Parts of Faces Using a Consensus of Exemplars , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Seyed-Ahmad Ahmadi,et al.  V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[9]  Yong Man Ro,et al.  A deep facial landmarks detection with facial contour and facial components constraint , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[10]  Qi Li,et al.  Fast multi-view face alignment via multi-task auto-encoders , 2017, 2017 IEEE International Joint Conference on Biometrics (IJCB).

[11]  Rama Chellappa,et al.  HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Thomas S. Huang,et al.  Interactive Facial Feature Localization , 2012, ECCV.

[13]  Ioannis Patras,et al.  Privileged Information-Based Conditional Structured Output Regression Forest for Facial Point Detection , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Ioannis A. Kakadiaris,et al.  Joint Head Pose Estimation and Face Alignment Framework Using Global and Local CNN Features , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[15]  Pietro Perona,et al.  Robust Face Landmark Estimation under Occlusion , 2013, 2013 IEEE International Conference on Computer Vision.

[16]  Yuting Zhang,et al.  Unsupervised Discovery of Object Landmarks as Structural Representations , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Xiaoming Liu,et al.  Disentangled Representation Learning GAN for Pose-Invariant Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Yong Man Ro,et al.  Differential Generative Adversarial Networks: Synthesizing Non-linear Facial Variations with Limited Number of Training Data , 2017, ArXiv.

[19]  Cheng Li,et al.  Face alignment by coarse-to-fine shape searching , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Ran He,et al.  Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[22]  Stefanos Zafeiriou,et al.  300 Faces In-The-Wild Challenge: database and results , 2016, Image Vis. Comput..

[23]  Maja Pantic,et al.  Gauss-Newton Deformable Part Models for Face Alignment In-the-Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Jiwen Lu,et al.  Learning Deep Sharable and Structural Detectors for Face Alignment , 2017, IEEE Transactions on Image Processing.

[25]  Xiaoou Tang,et al.  Learning Deep Representation for Face Alignment with Auxiliary Attributes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Marek Kowalski,et al.  Deep Alignment Network: A Convolutional Neural Network for Robust Face Alignment , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[27]  Jiwen Lu,et al.  Multiscale recurrent regression networks for face alignment , 2017, Applied Informatics.

[28]  Jian Sun,et al.  Face Alignment by Explicit Shape Regression , 2012, International Journal of Computer Vision.

[29]  Cheng Cheng,et al.  A Deep Regression Architecture with Two-Stage Re-initialization for High Performance Facial Landmark Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Georgios Tzimiropoulos,et al.  How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks) , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Yizhou Wang,et al.  Face Detection with End-to-End Integration of a ConvNet and a 3D Model , 2016, ECCV.

[32]  Yihong Gong,et al.  Face alignment recurrent network , 2018, Pattern Recognit..

[33]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[34]  Stefanos Zafeiriou,et al.  Robust Discriminative Response Map Fitting with Constrained Local Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Shiguang Shan,et al.  Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment , 2014, ECCV.

[36]  Nicu Sebe,et al.  Recurrent Convolutional Face Alignment , 2016, ACCV.

[37]  Zhenan Sun,et al.  Combining Data-Driven and Model-Driven Methods for Robust Facial Landmark Detection , 2016, IEEE Transactions on Information Forensics and Security.