Accurate 3D face modeling and recognition from RGB-D stream in the presence of large pose changes

We propose a 3D face modeling and recognition system using an RGB-D stream in the presence of large pose changes. In the previous work, all facial data points are registered with a reference to improve the accuracy of 3D face model from a low-resolution depth sequence. This registration often fails when applied to non-frontal faces. It causes inaccurate 3D face models and poor performance of matching. We address this problem by pre-aligning each input face (`frontalization') before the registration, which avoids registration failures. For each frame, our method estimates the 3D face pose, assesses the quality of data, segments the facial region, frontalizes it, and performs an accurate registration with the previous 3D model. The 3D-3D recognition system using accurate 3D models from our method outperforms other face recognition systems and shows 100% rank 1 recognition accuracy on a dataset with 30 subjects.

[1]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Thomas S. Huang,et al.  Automatic construction of 3D human face models based on 2D images , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[3]  Olga R. P. Bellon,et al.  3D Face Reconstruction Using a Single or Multiple Views , 2010, 2010 20th International Conference on Pattern Recognition.

[4]  Jongsun Kim,et al.  Effective representation using ICA for face recognition robust to local distortion and partial occlusion , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Guodong Guo,et al.  Face recognition robust to head pose changes based on the RGB-D sensor , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[6]  Jongmoo Choi,et al.  Real-time 3D face identification from a depth camera , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[7]  Jongmoo Choi,et al.  Near laser-scan quality 3-D face reconstruction from a low-quality depth stream , 2015, Image Vis. Comput..

[8]  Jongmoo Choi,et al.  Identifying Noncooperative Subjects at a Distance Using Face Images and Inferred Three-Dimensional Face Models , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[9]  Ramakant Nevatia,et al.  Face recognition using deep multi-pose representations , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[10]  Ajmal S. Mian,et al.  Using Kinect for face recognition under varying poses, expressions, illumination and disguise , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[11]  Dieter Fox,et al.  DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  V. Lepetit,et al.  EPnP: An Accurate O(n) Solution to the PnP Problem , 2009, International Journal of Computer Vision.

[14]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Günther Greiner,et al.  Automatic reconstruction of personalized avatars from 3D face scans , 2011, Comput. Animat. Virtual Worlds.

[16]  Jongmoo Choi,et al.  Accurate 3D face reconstruction from weakly calibrated wide baseline images with profile contours , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.