Annotated face model-based alignment: a robust landmark-free pose estimation approach for 3D model registration

Registering a 3D facial model onto a 2D image is important for constructing pixel-wise correspondences between different facial images. The registration is based on a 3 $$\times $$× 4 dimensional projection matrix, which is obtained from pose estimation. Conventional pose estimation approaches employ facial landmarks to determine the coefficients inside the projection matrix and are sensitive to missing or incorrect landmarks. In this paper, a landmark-free pose estimation method is presented. The method can be used to estimate the matrix when facial landmarks are not available. Experimental results show that the proposed method outperforms several landmark-free pose estimation methods and achieves competitive accuracy in terms of estimating pose parameters. The method is also demonstrated to be effective as part of a 3D-aided face recognition pipeline (UR2D), whose rank-1 identification rate is competitive to the methods that use landmarks to estimate head pose.

[1]  Alberto Del Bimbo,et al.  Using 3D Models to Recognize 2D Faces in the Wild , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[2]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Hossein Mobahi,et al.  Toward a Practical Face Recognition System: Robust Alignment and Illumination by Sparse Representation , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Gee-Sern Hsu,et al.  Face Recognition across Poses Using a Single 3D Reference Model , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[5]  Ioannis A. Kakadiaris,et al.  Towards fitting a 3D dense facial model to a 2D image: A landmark-free approach , 2015, 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[6]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Stefanos Zafeiriou,et al.  From Pixels to Response Maps: Discriminative Image Filtering for Face Alignment in the Wild , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Ioannis A. Kakadiaris,et al.  Pose-robust face signature for multi-view face recognition , 2015, 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[9]  Ioannis A. Kakadiaris,et al.  UHDB11 Database for 3D-2D Face Recognition , 2013, PSIVT.

[10]  Wenhan Luo,et al.  Unified Face Analysis by Iterative Multi-output Random Forests , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Xiantong Zhen,et al.  Supervised descriptor learning for multi-output regression , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  J. Crowley,et al.  Estimating Face orientation from Robust Detection of Salient Facial Structures , 2004 .

[13]  Fernando De la Torre,et al.  Global supervised descent method , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Ioannis Patras,et al.  Random Subspace Supervised Descent Method for Regression Problems in Computer Vision , 2015, IEEE Signal Processing Letters.

[15]  Jean-Marc Odobez,et al.  Recognizing Visual Focus of Attention From Head Pose in Natural Meetings , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  Ioannis A. Kakadiaris,et al.  Bidirectional relighting for 3D-aided 2D face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[19]  Alice J. O'Toole,et al.  FRVT 2006 and ICE 2006 Large-Scale Experimental Results , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Marios Savvides,et al.  Sparse Feature Extraction for Pose-Tolerant Face Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Yun Fu,et al.  Head pose estimation: Classification or regression? , 2008, 2008 19th International Conference on Pattern Recognition.

[22]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[24]  Stefanos Zafeiriou,et al.  Incremental Face Alignment in the Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Ioannis A. Kakadiaris,et al.  Benchmarking 3D Pose Estimation for Face Recognition , 2014, 2014 22nd International Conference on Pattern Recognition.

[26]  Yangang Wang,et al.  Online modeling for realtime facial animation , 2013, ACM Trans. Graph..

[27]  Ioannis A. Kakadiaris,et al.  3D-2D face recognition with pose and illumination normalization , 2017, Comput. Vis. Image Underst..

[28]  Ioannis A. Kakadiaris,et al.  Rendering or normalization? An analysis of the 3D-aided pose-invariant face recognition , 2016, 2016 IEEE International Conference on Identity, Security and Behavior Analysis (ISBA).

[29]  Ira Kemelmacher-Shlizerman,et al.  Face reconstruction in the wild , 2011, 2011 International Conference on Computer Vision.

[30]  Stefanos Zafeiriou,et al.  Robust Discriminative Response Map Fitting with Constrained Local Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Phong V. Vu,et al.  A Fast Wavelet-Based Algorithm for Global and Local Image Sharpness Estimation , 2012, IEEE Signal Processing Letters.

[32]  Xin Geng,et al.  Head Pose Estimation Based on Multivariate Label Distribution , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[34]  Horst Bischof,et al.  Supervised local subspace learning for continuous head pose estimation , 2011, CVPR 2011.

[35]  Xiangyu Zhu,et al.  High-fidelity Pose and Expression Normalization for face recognition in the wild , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Hao Li,et al.  Realtime performance-based facial animation , 2011, ACM Trans. Graph..

[37]  Rama Chellappa,et al.  Growing Regression Forests by Classification: Applications to Object Pose Estimation , 2013, ECCV.

[38]  Sethuraman Panchanathan,et al.  Biased Manifold Embedding: A Framework for Person-Independent Head Pose Estimation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Alexei A. Efros,et al.  How Important Are "Deformable Parts" in the Deformable Parts Model? , 2012, ECCV Workshops.

[41]  Peter Robinson,et al.  Face Alignment Assisted by Head Pose Estimation , 2015, BMVC.

[42]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[43]  Shiguang Shan,et al.  CovGa: A novel descriptor based on symmetry of regions for head pose estimation , 2014, Neurocomputing.

[44]  Takeo Kanade,et al.  Dense 3D face alignment from 2D videos in real-time , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[45]  Chao Wang,et al.  Robust head pose estimation via supervised manifold learning , 2014, Neural Networks.