Face Synthesis and Recognition Using Disentangled Representation-Learning Wasserstein GAN

We propose the Disentangled Representation-learning Wasserstein GAN (DR-WGAN) trained on augmented data for face recognition and face synthesis across pose. We improve the state-of-the-art DR-GAN with the Wasserstein loss considered in the discriminator so that the generative and adversarial framework can be better trained. The improved training leads to better face disentanglement and synthesis. We also highlight the influences of imbalanced training data on the disentangled facial representation learning, and point out the difficulty of generating faces of extreme poses. We explore the recently proposed nonlinear 3D Morphable Model (3DMM) to augment the training data, and verify the contributions made by the learning on augmented data. Additionally, we also compare different data normalization schemes and reveal the benefit of using the group normalization. The proposed framework is verified through the experiments on benchmark databases, and compared with contemporary approaches for performance evaluation.

[1]  Xiaogang Wang,et al.  Multi-View Perceptron: a Deep Model for Learning Face Identity and View Representations , 2014, NIPS.

[2]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Ran Tao,et al.  Is Pose Really Solved? A Frontalization Study On Off-Angle Face Matching , 2019, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[5]  Fang Zhao,et al.  Dual-Agent GANs for Photorealistic and Identity Preserving Profile Face Synthesis , 2017, NIPS.

[6]  Rama Chellappa,et al.  Unconstrained face verification using deep CNN features , 2015, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[7]  Rama Chellappa,et al.  Fisher vector encoded deep convolutional features for unconstrained face verification , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[8]  Byron C. Wallace,et al.  Structured Representations for Reviews: Aspect-Based Variational Hidden Factor Models , 2018, ArXiv.

[9]  Gee-Sern Jison Hsu,et al.  Face recognition by facial attribute assisted network , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[10]  Anil K. Jain,et al.  Face Search at Scale , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Carlos D. Castillo,et al.  Triplet probabilistic embedding for face verification and clustering , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[12]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[13]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[14]  Xiaoming Liu,et al.  Nonlinear 3D Face Morphable Model , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Shengcai Liao,et al.  Learning Face Representation from Scratch , 2014, ArXiv.

[16]  Anil K. Jain,et al.  Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Tal Hassner,et al.  Effective face frontalization in unconstrained images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Carlos D. Castillo,et al.  Frontal to profile face verification in the wild , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[19]  Tieniu Tan,et al.  A Light CNN for Deep Face Representation With Noisy Labels , 2015, IEEE Transactions on Information Forensics and Security.

[20]  Gérard G. Medioni,et al.  Pose-Aware Face Recognition in the Wild , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Xiaoming Liu,et al.  Disentangled Representation Learning GAN for Pose-Invariant Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[23]  Stefanos Zafeiriou,et al.  Robust Statistical Face Frontalization , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Gözde B. Ünal,et al.  Patch-Based Image Inpainting with Generative Adversarial Networks , 2018, ArXiv.

[25]  Georgios Tzimiropoulos,et al.  How Far are We from Solving the 2D & 3D Face Alignment Problem? (and a Dataset of 230,000 3D Facial Landmarks) , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Xiangyu Zhu,et al.  Face Alignment in Full Pose Range: A 3D Total Solution , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Du-Sik Park,et al.  Rotating your face using multi-task deep neural network , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Dimitris N. Metaxas,et al.  Reconstruction-Based Disentanglement for Pose-Invariant Face Recognition , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).