3D face texture modeling from uncalibrated frontal and profile images

3D face modeling from 2D face images is of significant importance for face analysis, animation and recognition. Previous research on this topic mainly focused on 3D face modeling from a single 2D face image; however, a single face image can only provide a limited description of a 3D face. In many applications, for example, law enforcement, multi-view face images are usually captured for a subject during enrollment, which makes it desirable to build a 3D face texture model, given a pair of frontal and profile face images. We first determine the correspondence between un-calibrated frontal and profile face images through facial landmark alignment. An initial 3D face shape is then reconstructed from the frontal face image, followed by shape refinement utilizing the depth information provided by the profile image. Finally, face texture is extracted by mapping the frontal face image on the recovered 3D face shape. The proposed method is utilized for 2D face recognition in two scenarios: (i) normalization of probe image, and (ii) enhancing the representation capability of gallery set. Experimental results comparing the proposed method with a state-of-the-art commercial face matcher and densely sampled LBP on a subset of the FERET database show the effectiveness of the proposed 3D face texture model.

[1]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  A-Nasser Ansari,et al.  Automatic facial feature extraction and 3D face modeling using two orthogonal views with application to 3D face recognition , 2005, Pattern Recognit..

[3]  Yajie Tian,et al.  Handbook of face recognition , 2003 .

[4]  Haitao Wang,et al.  Face recognition under varying lighting conditions using self quotient image , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[5]  Tomaso A. Poggio,et al.  Linear Object Classes and Image Synthesis From a Single Example Image , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[7]  Yiying Tong,et al.  Age-Invariant Face Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Nathan Faggian,et al.  3D Morphable Model fitting from multiple views , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[9]  Anil K. Jain,et al.  Handbook of Face Recognition, 2nd Edition , 2011 .

[10]  Wen Gao,et al.  Lighting Aware Preprocessing for Face Recognition across Varying Illumination , 2010, ECCV.

[11]  Fred Nicolls,et al.  Locating Facial Features with an Extended Active Shape Model , 2008, ECCV.

[12]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[13]  Rama Chellappa,et al.  Illumination-insensitive face recognition using symmetric shape-from-shading , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  David Beymer,et al.  Face recognition from one example view , 1995, Proceedings of IEEE International Conference on Computer Vision.

[15]  Ronen Basri,et al.  Comparing images under variable illumination , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[16]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[17]  Arun Ross,et al.  Handbook of Multibiometrics , 2006, The Kluwer international series on biometrics.

[18]  Alice J. O'Toole,et al.  FRVT 2006 and ICE 2006 Large-Scale Experimental Results , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  T. Kanade,et al.  Combining Models and Exemplars for Face Recognition: An Illuminating Example , 2001 .

[22]  C. Beumier,et al.  3D Face Recognition , 2004, 2006 IEEE International Conference on Industrial Technology.

[23]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[24]  Edwin R. Hancock,et al.  Acquiring height data from a single image of a face using local shape indicators , 2006, Comput. Vis. Image Underst..

[25]  Thomas Vetter,et al.  Synthesis of Novel Views from a Single Face Image , 1998, International Journal of Computer Vision.

[26]  Olivier D. Faugeras,et al.  Shape From Shading , 2006, Handbook of Mathematical Models in Computer Vision.

[27]  Tat-Seng Chua,et al.  Morphable face reconstruction with multiple images , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[28]  Marios Savvides,et al.  Rapid 3D face modeling using a frontal face and a profile face for accurate 2D pose synthesis , 2011, Face and Gesture 2011.

[29]  Ralph Gross,et al.  An Image Preprocessing Algorithm for Illumination Invariant Face Recognition , 2003, AVBPA.

[30]  Wen Gao,et al.  Efficient 3D reconstruction for face recognition , 2005, Pattern Recognit..

[31]  Rama Chellappa,et al.  Symmetric Shape-from-Shading Using Self-ratio Image , 2001, International Journal of Computer Vision.

[32]  Wen Gao,et al.  Separability Oriented Preprocessing for Illumination-Insensitive Face Recognition , 2012, ECCV.

[33]  Lijun Yin,et al.  Constructing a 3D individualized head model from two orthogonal views , 1996, The Visual Computer.

[34]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Lei Zhang,et al.  Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Yongsheng Gao,et al.  Recognizing Rotated Faces From Frontal and Side Views: An Approach Toward Effective Use of Mugshot Databases , 2008, IEEE Transactions on Information Forensics and Security.

[37]  Paul A. Griffin,et al.  Statistical Approach to Shape from Shading: Reconstruction of Three-Dimensional Face Surfaces from Single Two-Dimensional Images , 1996, Neural Computation.

[38]  Patrick Grother,et al.  Face Recognition Vendor Test (FRVT) , 2014 .

[39]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[40]  Ioannis A. Kakadiaris,et al.  UR3D-C: Linear dimensionality reduction for efficient 3D face recognition , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[41]  Amnon Shashua,et al.  The Quotient Image: Class-Based Re-Rendering and Recognition with Varying Illuminations , 2001, IEEE Trans. Pattern Anal. Mach. Intell..