3D facial landmark localization using texture regression via conformal mapping

We propose a new 3D facial landmark detection method based on regression method on 2D image.Our method combines both 3D shape and 2D texture information.Our method is robust to pose and expression. 3D facial alignment typically requires the accurate estimate of facial landmarks. Most existing landmark detection methods use geometry characterization or resort to regression algorithms performed on point clouds or range images. A method combining both 3D geometry information and 2D texture has rarely been investigated. In this paper, we propose a novel 3D facial landmark localization algorithm, based on conformal geometric mapping, that can convert a 3D model to 2D using both geometry and texture information. Then a two-layers-regression method is used to improve the stability of landmark localization on the 2D geometry images. This method is impervious to pose changes and robust with respect to changes in expression. We evaluated the proposed approach on publicly available datasets and demonstrate how the use of 2D regression methods boosts the robustness and accuracy of 3D facial landmark localization.

[1]  Chun Chen,et al.  Robust 3D Face Landmark Localization Based on Local Coordinate Coding , 2014, IEEE Transactions on Image Processing.

[2]  Anil K. Jain,et al.  Matching 2.5D face scans to 3D models , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Jürgen Beyerer,et al.  Fast, robust and automatic 3D face model reconstruction from videos , 2014, 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[4]  Dongmei Zhang,et al.  Harmonic maps and their applications in surface matching , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[5]  Lijun Yin,et al.  Tracking Vertex Flow and Model Adaptation for Three-Dimensional Spatiotemporal Face Analysis , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[6]  L. Akarun,et al.  3D Facial Landmarking under Expression, Pose, and Occlusion Variations , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[7]  Luc Van Gool,et al.  Real time 3D face alignment with Random Forests-based Active Appearance Models , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[8]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[9]  Ramesh C. Jain,et al.  Segmentation through Variable-Order Surface Fitting , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Ioannis A. Kakadiaris,et al.  Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Shiguang Shan,et al.  Cascaded Shape Space Pruning for Robust Facial Landmark Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[12]  Mohammed Bennamoun,et al.  An Efficient Multimodal 2D-3D Hybrid Approach to Automatic Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Timothy F. Cootes,et al.  Interpreting face images using active appearance models , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[14]  Maurício Pamplona Segundo,et al.  Automatic Face Segmentation and Facial Landmark Detection in Range Images , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Jun Wang,et al.  A 3D facial expression database for facial behavior research , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[16]  Raimondo Schettini,et al.  3D face detection using curvature analysis , 2006, Pattern Recognit..

[17]  Wang Chengzhang,et al.  BJUT-3D Large Scale 3D Face Database and Information Processing , 2009 .

[18]  Παναγιώτης Β. Περάκης.  Landmark detection for unconstrained face recognition , 2013 .

[19]  Yichen Wei,et al.  Face alignment by Explicit Shape Regression , 2012, CVPR.

[20]  Bernadette Dorizzi,et al.  3D Active Shape Model for Automatic Facial Landmark Location Trained with Automatically Generated Landmark Points , 2010, 2010 20th International Conference on Pattern Recognition.

[21]  W. C. Chen,et al.  3D face authentication by mutual coupled 3D and 2D feature extraction , 2006, ACM-SE 44.

[22]  Ioannis A. Kakadiaris,et al.  Automatic 3D Facial Region Retrieval from Multi-pose Facial Datasets , 2009, 3DOR@Eurographics.

[23]  Ioannis A. Kakadiaris,et al.  3D Facial Landmark Detection under Large Yaw and Expression Variations , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Paul F. Whelan,et al.  3-D Facial Landmark Localization With Asymmetry Patterns and Shape Regression from Incomplete Local Features , 2015, IEEE Transactions on Cybernetics.

[25]  Ioannis A. Kakadiaris,et al.  Feature fusion for facial landmark detection , 2014, Pattern Recognit..

[26]  Chitra Dorai,et al.  COSMOS - A Representation Scheme for 3D Free-Form Objects , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Jian Sun,et al.  Face Alignment at 3000 FPS via Regressing Local Binary Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Alan C. Bovik,et al.  Passive Multimodal 2-D+3-D Face Recognition Using Gabor Features and Landmark Distances , 2011, IEEE Transactions on Information Forensics and Security.

[29]  Alberto Del Bimbo,et al.  3D facial expression recognition using SIFT descriptors of automatically detected keypoints , 2011, The Visual Computer.

[30]  Liming Chen,et al.  Towards 3D Face Recognition in the Real: A Registration-Free Approach Using Fine-Grained Matching of 3D Keypoint Descriptors , 2014, International Journal of Computer Vision.

[31]  William J. Christmas,et al.  Cascaded Collaborative Regression for Robust Facial Landmark Detection Trained Using a Mixture of Synthetic and Real Images With Dynamic Weighting , 2015, IEEE Transactions on Image Processing.

[32]  Maja Pantic,et al.  Fully automatic facial feature point detection using Gabor feature based boosted classifiers , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[33]  Sen Wang,et al.  3D Surface Matching and Recognition Using Conformal Geometry , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[34]  Andrea Cavallaro,et al.  3-D Face Detection, Landmark Localization, and Registration Using a Point Distribution Model , 2009, IEEE Transactions on Multimedia.