Reconstructing High-Resolution Face Models From Kinect Depth Sequences

Performing face recognition across 3D scans with different resolution is now attracting an increasing interest thanks to the introduction of a new generation of depth cameras, capable of acquiring color/depth images over time. In fact, these devices acquire and provide depth data with much lower resolution compared with the 3D high-resolution scanners typically used for face recognition applications. If data are acquired without user cooperation, the problem is even more challenging, and the gap of resolution between probe and gallery scans can yield to a severe loss in terms of recognition accuracy. Based on these premises, we propose a method to build a higher resolution 3D face model from 3D data acquired by a low-resolution scanner. This face model is built using data acquired when a person passes in front of the scanner, without assuming any particular cooperation. The 3D data are registered and filtered by combining a model of the expected distribution of the acquisition error with a variant of the lowess method to remove outliers and build the final face model. The proposed approach is evaluated in terms of accuracy of face reconstruction and face recognition.

[1]  Sebastian Thrun,et al.  LidarBoost: Depth superresolution for ToF 3D shape scanning , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Maurício Pamplona Segundo,et al.  Continuous 3D Face Authentication Using RGB-D Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[3]  William G. Jacoby.  Loess: a nonparametric, graphical tool for depicting relationships between variables , 2000 .

[4]  Alberto Del Bimbo,et al.  Superfaces: A Super-Resolution Model for 3D Faces , 2012, ECCV Workshops.

[5]  Samarth Bharadwaj,et al.  On RGB-D face recognition using Kinect , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[6]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[7]  Sébastien Marcel,et al.  Spoofing in 2D face recognition with 3D masks and anti-spoofing with Kinect , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[8]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[9]  Fabio Roli,et al.  Multimodal Person Reidentification Using RGB-D Cameras , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Jongmoo Choi,et al.  Laser scan quality 3-D face modeling using a low-cost depth camera , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[11]  Alberto Del Bimbo,et al.  Face Recognition by Super-Resolved 3D Models From Consumer Depth Cameras , 2014, IEEE Transactions on Information Forensics and Security.

[12]  Shu Liang,et al.  3D Face Hallucination from a Single Depth Frame , 2014, 2014 2nd International Conference on 3D Vision.

[13]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[14]  Ajmal S. Mian,et al.  Using Kinect for face recognition under varying poses, expressions, illumination and disguise , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[15]  W. Cleveland.  Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[16]  Hassen Drira,et al.  3D Face Recognition under Expressions, Occlusions, and Pose Variations , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Paolo Frasconi,et al.  Large-scale automated identification of mouse brain cells in confocal light sheet microscopy images , 2014, Bioinform..

[18]  Dieter Fox,et al.  DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Björn E. Ottersten,et al.  Real-time non-rigid multi-frame depth video super-resolution , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[20]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[21]  Zhaohui Wu,et al.  Learning-based super-resolution of 3D face model , 2005, IEEE International Conference on Image Processing 2005.

[22]  Alberto Del Bimbo,et al.  Sparse Matching of Salient Facial Curves for Recognition of 3-D Faces With Missing Parts , 2013, IEEE Transactions on Information Forensics and Security.

[23]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[24]  Stefan Schaal,et al.  Local Dimensionality Reduction for Non-Parametric Regression , 2009, Neural Processing Letters.

[25]  Alberto Del Bimbo,et al.  The florence 2D/3D hybrid face dataset , 2011, J-HGBU '11.

[26]  Tieniu Tan,et al.  Combining local features for robust nose location in 3D facial data , 2006, Pattern Recognit. Lett..

[27]  Jean-Luc Dugelay,et al.  KinectFaceDB: A Kinect Database for Face Recognition , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[28]  Jongmoo Choi,et al.  Real-time 3D face identification from a depth camera , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[29]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[30]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Ioannis A. Kakadiaris,et al.  Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Maurício Pamplona Segundo,et al.  Real-time scale-invariant face detection on range images , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[33]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time dynamic 3D surface reconstruction and interaction , 2011, SIGGRAPH '11.

[35]  Yasushi Yagi,et al.  A Probabilistic Method for Aligning and Merging Range Images with Anisotropic Error Distribution , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[36]  Jongmoo Choi,et al.  Comparing strategies for 3D face recognition from a 3D sensor , 2013, 2013 IEEE RO-MAN.

[37]  Zhaohui Wu,et al.  Super-Resolution of 3D Face , 2006, ECCV.

[38]  John A. Williams,et al.  Multiple view surface registration with error modeling and analysis , 2000, Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101).