On Mugshot-based Arbitrary View Face Recognition

Despite the wide usage of mugshot images in forensic applications, they are underutilized in existing automated face recognition systems. In this paper, we propose a novel mugshot-based arbitrary view face recognition method. Our approach reconstructs full 3D faces via cascaded regression in shape space with efficient seamless texture recovery. Unlike existing methods, it makes full use of the frontal and profile views available in mugshot images, and thus generates accurate and realistic 3D faces. Multi-view face images are synthesized from the reconstructed 3D faces to enlarge the gallery so that arbitrary view faces can be better recognized. Evaluation experiments were conducted on BFM and Multi-PIE databases by using state-of-the-art deep learning (DL) based face matchers. The results demonstrate the effectiveness of our proposed method and show that DL-based face matchers can benefit from mugshot images and the reconstructed 3D faces, especially for recognizing large off-angle faces.

[1]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Bhiksha Raj,et al.  SphereFace: Deep Hypersphere Embedding for Face Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Feng Liu,et al.  Joint Face Alignment and 3D Face Reconstruction , 2016, ECCV.

[4]  Jürgen Beyerer,et al.  Fast, robust and automatic 3D face model reconstruction from videos , 2014, 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[5]  Sami Romdhani,et al.  Estimating 3D shape and texture using pixel intensity, edges, specular highlights, texture constraints and a prior , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[6]  Xiangyu Zhu,et al.  Robust 3D Morphable Model Fitting by Sparse SIFT Flow , 2014, 2014 22nd International Conference on Pattern Recognition.

[7]  Yongsheng Gao,et al.  Recognizing Rotated Faces From Frontal and Side Views: An Approach Toward Effective Use of Mugshot Databases , 2008, IEEE Transactions on Information Forensics and Security.

[8]  Sami Romdhani,et al.  A 3D Face Model for Pose and Illumination Invariant Face Recognition , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[9]  Anil K. Jain,et al.  3D face texture modeling from uncalibrated frontal and profile images , 2012, 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[10]  Zeev Farbman,et al.  Convolution pyramids , 2011, ACM Trans. Graph..

[11]  Edwin R. Hancock,et al.  Seamless texture stitching on a 3D mesh by poisson blending in patches , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[12]  Xiangyu Zhu,et al.  Face Alignment in Full Pose Range: A 3D Total Solution , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[14]  Edwin R. Hancock,et al.  Symmetry‐Aware Mesh Segmentation into Uniform Overlapping Patches , 2017, Comput. Graph. Forum.

[15]  Feng Liu,et al.  On 3D face reconstruction via cascaded regression in shape space , 2015, Frontiers of Information Technology & Electronic Engineering.

[16]  Shiguang Shan,et al.  LDF-Net: Learning a Displacement Field Network for Face Recognition across Pose , 2017, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[17]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[18]  Tieniu Tan,et al.  A Light CNN for Deep Face Representation With Noisy Labels , 2015, IEEE Transactions on Information Forensics and Security.

[19]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Xiangyu Zhu,et al.  High-fidelity Pose and Expression Normalization for face recognition in the wild , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Gerhard Rigoll,et al.  Recognition of face profiles from the mugshot database using a hybrid connectionist/HMM approach , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[22]  Qijun Zhao,et al.  A Novel Approach to Mugshot Based Arbitrary View Face Recognition , 2016 .

[23]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.