Increasing 3D Resolution of Kinect Faces

Performing face recognition across 3D scans of different resolution is now attracting an increasing interest thanks to the introduction of a new generation of depth cameras, capable of acquiring color/depth images over time. However, these devices have still a much lower resolution than the 3D high-resolution scanners typically used for face recognition applications. Due to this, comparing low- and high-resolution scans can be misleading. Based on these considerations, in this paper we define an approach for reconstructing a higher-resolution 3D face model from a sequence of low-resolution 3D scans. The proposed solution uses the scaled ICP algorithm to align the low-resolution scans with each other, and estimates the value of the high-resolution 3D model through a 2D Box-spline approximation. The approach is evaluated on the The Florence face dataset that collects high- and low-resolution data for about 50 subjects. Measures of the quality of the reconstructed models with respect to high-resolution scans and in comparison with two alternative techniques, demonstrate the viability of the proposed solution.

[1]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[2]  Nanning Zheng,et al.  Scaling iterative closest point algorithm for registration of m-D point sets , 2010, J. Vis. Commun. Image Represent..

[3]  Jean-Luc Dugelay,et al.  An Efficient LBP-Based Descriptor for Facial Depth Images Applied to Gender Recognition Using RGB-D Face Data , 2012, ACCV Workshops.

[4]  Tieniu Tan,et al.  Combining local features for robust nose location in 3D facial data , 2006, Pattern Recognit. Lett..

[5]  Ajmal S. Mian,et al.  Using Kinect for face recognition under varying poses, expressions, illumination and disguise , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[6]  Mehran Ebrahimi,et al.  Multi-Frame Super-Resolution with No Explicit Motion Estimation , 2008, IPCV.

[7]  Ioannis A. Kakadiaris,et al.  Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Georg Zimmermann,et al.  Scalar multivariate subdivision schemes and box splines , 2011, Comput. Aided Geom. Des..

[9]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[11]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[12]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time dynamic 3D surface reconstruction and interaction , 2011, SIGGRAPH '11.

[13]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  Maurício Pamplona Segundo,et al.  Real-time scale-invariant face detection on range images , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[15]  Jongmoo Choi,et al.  Comparing strategies for 3D face recognition from a 3D sensor , 2013, 2013 IEEE RO-MAN.

[16]  Jongmoo Choi,et al.  Laser scan quality 3-D face modeling using a low-cost depth camera , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[17]  Hassen Drira,et al.  3D Face Recognition under Expressions, Occlusions, and Pose Variations , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Jongmoo Choi,et al.  Real-time 3D face identification from a depth camera , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[19]  Zhaohui Wu,et al.  Learning-based super-resolution of 3D face model , 2005, IEEE International Conference on Image Processing 2005.

[20]  Sébastien Marcel,et al.  Spoofing in 2D face recognition with 3D masks and anti-spoofing with Kinect , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[21]  Maurício Pamplona Segundo,et al.  Continuous 3D Face Authentication Using RGB-D Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[22]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[23]  Alberto Del Bimbo,et al.  Superfaces: A Super-Resolution Model for 3D Faces , 2012, ECCV Workshops.

[24]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[25]  Zhaohui Wu,et al.  Super-Resolution of 3D Face , 2006, ECCV.

[26]  Samarth Bharadwaj,et al.  On RGB-D face recognition using Kinect , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[27]  Sebastian Thrun,et al.  LidarBoost: Depth superresolution for ToF 3D shape scanning , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.