Face Recognition by Super-Resolved 3D Models From Consumer Depth Cameras

Face recognition based on the analysis of 3D scans has been an active research subject over the last few years. However, the impact of the resolution of 3D scans on the recognition process has not been addressed explicitly, yet being an element of primal importance to enable the use of the new generation of consumer depth cameras for biometric purposes. In fact, these devices perform depth/color acquisition over time at standard frame-rate, but with a low resolution compared to the 3D scanners typically used for acquiring 3D faces in recognition applications. Motivated by these considerations, in this paper, we define a super-resolution approach for 3D faces by which a sequence of low-resolution 3D face scans is processed to extract a higher resolution 3D face model. The proposed solution relies on the scaled iterative closest point procedure to align the low-resolution scans with each other, and estimates the value of the high-resolution 3D model through a 2D box-spline functions approximation. To evaluate the approach, we built-and made it publicly available-the Florence Superface dataset that collects high-resolution and low-resolution data for about 50 different persons. Qualitative and quantitative results are reported to demonstrate the accuracy of the proposed solution, also in comparison with alternative techniques.

[1]  Sébastien Marcel,et al.  Spoofing in 2D face recognition with 3D masks and anti-spoofing with Kinect , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[2]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time dynamic 3D surface reconstruction and interaction , 2011, SIGGRAPH '11.

[4]  Michael Unser,et al.  B-spline signal processing. I. Theory , 1993, IEEE Trans. Signal Process..

[5]  Alberto Del Bimbo,et al.  Sparse Matching of Salient Facial Curves for Recognition of 3-D Faces With Missing Parts , 2013, IEEE Transactions on Information Forensics and Security.

[6]  Ioannis A. Kakadiaris,et al.  Using Facial Symmetry to Handle Pose Variations in Real-World 3D Face Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Georg Zimmermann,et al.  Scalar multivariate subdivision schemes and box splines , 2011, Comput. Aided Geom. Des..

[8]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[9]  Jongmoo Choi,et al.  Comparing strategies for 3D face recognition from a 3D sensor , 2013, 2013 IEEE RO-MAN.

[10]  Raimondo Schettini,et al.  Gappy PCA Classification for Occlusion Tolerant 3D Face Detection , 2009, Journal of Mathematical Imaging and Vision.

[11]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[12]  Akram Aldroubi,et al.  B-spline signal processing. II. Efficiency design and applications , 1993, IEEE Trans. Signal Process..

[13]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  Maurício Pamplona Segundo,et al.  Real-time scale-invariant face detection on range images , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[15]  Samarth Bharadwaj,et al.  On RGB-D face recognition using Kinect , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[16]  Alberto Del Bimbo,et al.  The florence 2D/3D hybrid face dataset , 2011, J-HGBU '11.

[17]  Jongmoo Choi,et al.  Real-time 3D face identification from a depth camera , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[18]  Luc Van Gool,et al.  Random Forests for Real Time 3D Face Analysis , 2012, International Journal of Computer Vision.

[19]  L. Spreeuwers.  Fast and Accurate 3 D Face Recognition Using Registration to an Intrinsic Coordinate System and Fusion of Multiple Region Classifiers , 2011 .

[20]  Arman Savran,et al.  Bosphorus Database for 3D Face Analysis , 2008, BIOID.

[21]  Roger Y. Tsai,et al.  Multiframe image restoration and registration , 1984 .

[22]  Nanning Zheng,et al.  Scaling iterative closest point algorithm for registration of m-D point sets , 2010, J. Vis. Commun. Image Represent..

[23]  Jean-Luc Dugelay,et al.  An Efficient LBP-Based Descriptor for Facial Depth Images Applied to Gender Recognition Using RGB-D Face Data , 2012, ACCV Workshops.

[24]  David A. Maluf,et al.  Bayesian super-resolved surface reconstruction from images , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[25]  Hassen Drira,et al.  3D Face Recognition under Expressions, Occlusions, and Pose Variations , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Maurício Pamplona Segundo,et al.  Continuous 3D Face Authentication Using RGB-D Cameras , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[27]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Tieniu Tan,et al.  Combining local features for robust nose location in 3D facial data , 2006, Pattern Recognit. Lett..

[29]  Luuk J. Spreeuwers,et al.  Fast and Accurate 3D Face Recognition , 2011, International Journal of Computer Vision.

[30]  Xiaoou Tang,et al.  Robust 3D Face Recognition by Local Shape Difference Boosting , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Alberto Del Bimbo,et al.  3D Face Recognition Using Isogeodesic Stripes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Ajmal S. Mian,et al.  Using Kinect for face recognition under varying poses, expressions, illumination and disguise , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[33]  Mehran Ebrahimi,et al.  Multi-Frame Super-Resolution with No Explicit Motion Estimation , 2008, IPCV.

[34]  Alberto Del Bimbo,et al.  Superfaces: A Super-Resolution Model for 3D Faces , 2012, ECCV Workshops.

[35]  Luc Van Gool,et al.  Real Time Head Pose Estimation from Consumer Depth Cameras , 2011, DAGM-Symposium.

[36]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[37]  Jongmoo Choi,et al.  Laser scan quality 3-D face modeling using a low-cost depth camera , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[38]  Russell C. Hardie,et al.  Joint MAP registration and high-resolution image estimation using a sequence of undersampled images , 1997, IEEE Trans. Image Process..

[39]  Zhaohui Wu,et al.  Super-Resolution of 3D Face , 2006, ECCV.

[40]  Zhaohui Wu,et al.  Learning-based super-resolution of 3D face model , 2005, IEEE International Conference on Image Processing 2005.

[41]  Sebastian Thrun,et al.  LidarBoost: Depth superresolution for ToF 3D shape scanning , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.