Robust Statistical Face Frontalization

Recently, it has been shown that excellent results can be achieved in both facial landmark localization and pose-invariant face recognition. These breakthroughs are attributed to the efforts of the community to manually annotate facial images in many different poses and to collect 3D facial data. In this paper, we propose a novel method for joint frontal view reconstruction and landmark localization using a small set of frontal images only. By observing that the frontal facial image is the one having the minimum rank of all different poses, an appropriate model which is able to jointly recover the frontalized version of the face as well as the facial landmarks is devised. To this end, a suitable optimization problem, involving the minimization of the nuclear norm and the matrix l1 norm is solved. The proposed method is assessed in frontal face reconstruction, face landmark localization, pose-invariant face recognition, and face verification in unconstrained conditions. The relevant experiments have been conducted on 8 databases. The experimental results demonstrate the effectiveness of the proposed method in comparison to the state-of-the-art methods for the target problems.

[1]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Stefanos Zafeiriou,et al.  A survey on face detection in the wild: Past, present and future , 2015, Comput. Vis. Image Underst..

[3]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[4]  Yi Ma,et al.  TILT: Transform Invariant Low-Rank Textures , 2010, ACCV 2010.

[5]  Stefanos Zafeiriou,et al.  RAPS: Robust and Efficient Automatic Construction of Person-Specific Deformable Models , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Tal Hassner,et al.  Effective face frontalization in unconstrained images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Stefanos Zafeiriou,et al.  Menpo: A Comprehensive Platform for Parametric Image Alignment and Visual Deformable Models , 2014, ACM Multimedia.

[9]  Xiaogang Wang,et al.  Deep Learning Identity-Preserving Face Space , 2013, 2013 IEEE International Conference on Computer Vision.

[10]  Xin Liu,et al.  Morphable Displacement Field Based Image Matching for Face Recognition across Pose , 2012, ECCV.

[11]  Thomas S. Huang,et al.  Interactive Facial Feature Localization , 2012, ECCV.

[12]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[13]  Stefanos Zafeiriou,et al.  Feature-Based Lucas–Kanade and Active Appearance Models , 2015, IEEE Transactions on Image Processing.

[14]  Amit R.Sharma,et al.  Face Photo-Sketch Synthesis and Recognition , 2012 .

[15]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Stefanos Zafeiriou,et al.  Robust Discriminative Response Map Fitting with Constrained Local Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Xiaogang Wang,et al.  Coupled information-theoretic encoding for face photo-sketch recognition , 2011, CVPR 2011.

[18]  Honglak Lee,et al.  Learning to Align from Scratch , 2012, NIPS.

[19]  Stefanos Zafeiriou,et al.  Generalised Scalable Robust Principal Component Analysis , 2014, BMVC.

[20]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[21]  Weiwei Zhang,et al.  Cat Head Detection - How to Effectively Exploit Shape and Texture Features , 2008, ECCV.

[22]  Stefanos Zafeiriou,et al.  Automatic Construction of Deformable Models In-the-Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Stefanos Zafeiriou,et al.  A Semi-automatic Methodology for Facial Landmark Annotation , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[24]  Constantine Kotropoulos,et al.  Music Genre Classification via Joint Sparse Low-Rank Representation of Audio Features , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[25]  Rama Chellappa,et al.  Pose-Invariant Face Recognition Using Markov Random Fields , 2013, IEEE Transactions on Image Processing.

[26]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.

[27]  Sridha Sridharan,et al.  Deformable face ensemble alignment with robust grouped-L1 anchors , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[28]  Stefanos Zafeiriou,et al.  Robust Canonical Time Warping for the Alignment of Grossly Corrupted Sequences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Michael J. Jones,et al.  Fully automatic pose-invariant face recognition via 3D pose normalization , 2011, 2011 International Conference on Computer Vision.

[30]  Jian Sun,et al.  Face Alignment at 3000 FPS via Regressing Local Binary Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Stefanos Zafeiriou,et al.  Subspace Learning from Image Gradient Orientations , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  John Wright,et al.  RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[33]  Stan Z. Li,et al.  Towards Pose Robust Face Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Wen Gao,et al.  Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[35]  Simon Baker,et al.  Active Appearance Models Revisited , 2004, International Journal of Computer Vision.

[36]  Stefanos Zafeiriou,et al.  300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[37]  Josef Kittler,et al.  Class-Specific Kernel Fusion of Multiple Descriptors for Face Verification Using Multiscale Binarised Statistical Image Features , 2014, IEEE Transactions on Information Forensics and Security.

[38]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[39]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[40]  Hossein Mobahi,et al.  Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  David J. Kriegman,et al.  Localizing Parts of Faces Using a Consensus of Exemplars , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Maja Pantic,et al.  Generic Active Appearance Models Revisited , 2012, ACCV.

[44]  Gang Hua,et al.  Eigen-PEP for Video Face Recognition , 2014, ACCV.

[45]  Andrew Zisserman,et al.  Fisher Vector Faces in the Wild , 2013, BMVC.

[46]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[47]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.