Minimizing Illumination Differences for 3D to 2D Face Recognition Using Lighting Maps

Asymmetric 3D to 2D face recognition has gained attention from the research community since the real-world application of 3D to 3D recognition is limited by the unavailability of inexpensive 3D data acquisition equipment. A 3D to 2D face recognition system explicitly relies on 3D facial data to account for uncontrolled image conditions related to head pose or illumination. We build upon such a system, which matches relit gallery textures with pose-normalized probe images, using the gallery facial meshes. The relighting process, however, is based on an assumption of indoor lighting conditions and limits recognition performance on outdoor images. In this paper, we propose a novel method for minimizing illumination difference by unlighting a 3D face texture via albedo estimation using lighting maps. The algorithm is evaluated on challenging databases (UHDB30, UHDB11, FRGC v2.0) with drastic lighting and pose variations. The experimental results demonstrate the robustness of our method for estimating the albedo from both indoor and outdoor captured images, and the effectiveness and efficiency for illumination normalization in face recognition.

[1]  Ioannis A. Kakadiaris,et al.  Illumination Normalization Using Self-lighting Ratios for 3D2D Face Recognition , 2012, ECCV Workshops.

[2]  Ioannis A. Kakadiaris,et al.  Evaluation of 3D Face Recognition in the presence of facial expressions: an Annotated Deformable Model approach , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[3]  Ioannis A. Kakadiaris,et al.  Facial landmark detection in uncontrolled conditions , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[4]  Bui Tuong Phong.  Illumination for computer generated pictures , 1975, Commun. ACM.

[5]  Chong-Ho Choi,et al.  Robust albedo estimation from a facial image with cast shadow , 2011, 2011 18th IEEE International Conference on Image Processing.

[6]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Ioannis A. Kakadiaris,et al.  Accurate Landmarking of Three-Dimensional Facial Data in the Presence of Facial Expressions and Occlusions Using a Three-Dimensional Statistical Facial Feature Model , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[8]  Rama Chellappa,et al.  Robust Estimation of Albedo for Illumination-invariant Matching and Shape Recovery , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[9]  Philip L. Worthington,et al.  Re-illuminating single images using Albedo estimation , 2005, Pattern Recognit..

[10]  Liming Chen,et al.  Oriented Gradient Maps based automatic asymmetric 3D-2D face recognition , 2012, 2012 5th IAPR International Conference on Biometrics (ICB).

[11]  Pascal Fua,et al.  Recovery of Reflectances and Varying Illuminants from Multiple Views , 2002, ECCV.

[12]  Zihan Zhou,et al.  Nearest-Subspace Patch Matching for face recognition under varying pose and illumination , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[13]  Meng Joo Er,et al.  Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[14]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[15]  Ioannis A. Kakadiaris,et al.  Bidirectional relighting for 3D-aided 2D face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Andrea F. Abate,et al.  2D and 3D face recognition: A survey , 2007, Pattern Recognit. Lett..

[17]  Tatsuo Arai,et al.  Illumination Normalization for Outdoor Face Recognition by Using Ayofa-Filters , 2011 .

[18]  Patrick J. Flynn,et al.  A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition , 2006, Comput. Vis. Image Underst..

[19]  Ahmad Ayatollahi,et al.  A Survey of 3 D Face Recognition Methods , 2013 .

[20]  Edwin R. Hancock,et al.  Recovering Facial Shape Using a Statistical Model of Surface Normal Direction , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Faisal R. Al-Osaimi,et al.  Illumination normalization of facial images by reversing the process of image formation , 2011, Machine Vision and Applications.

[22]  Rama Chellappa,et al.  Pose-robust albedo estimation from a single image , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Rama Chellappa,et al.  Appearance Characterization of Linear Lambertian Objects, Generalized Photometric Stereo, and Illumination-Invariant Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Chu-Song Chen,et al.  Intrinsic Illumination Subspace for Lighting Insensitive Face Recognition , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Alan C. Bovik,et al.  Passive Multimodal 2-D+3-D Face Recognition Using Gabor Features and Landmark Distances , 2011, IEEE Transactions on Information Forensics and Security.

[26]  Stefanos Zafeiriou,et al.  Robust FFT-Based Scale-Invariant Image Registration with Image Gradients , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Ioannis A. Kakadiaris,et al.  Three-Dimensional Face Recognition in the Presence of Facial Expressions: An Annotated Deformable Model Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Ioannis A. Kakadiaris,et al.  Illumination alignment using lighting ratio: Application to 3D-2D face recognition , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[29]  M. Gross,et al.  Analysis of human faces using a measurement-based skin reflectance model , 2006, ACM Trans. Graph..

[30]  Ronen Basri,et al.  Lambertian reflectance and linear subspaces , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[31]  Alice J. O'Toole,et al.  FRVT 2006 and ICE 2006 Large-Scale Experimental Results , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Thomas Vetter,et al.  Face Recognition Based on Fitting a 3D Morphable Model , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Lei Zhang,et al.  Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Marios Savvides,et al.  Unconstrained Pose-Invariant Face Recognition Using 3D Generic Elastic Models , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Jian-Jun Zhang,et al.  Self quotient image for face recognition , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[36]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[37]  Qi Tian,et al.  Multiview Face Recognition: From TensorFace to V-TensorFace and K-TensorFace , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[38]  David J. Kriegman,et al.  What Is the Set of Images of an Object Under All Possible Illumination Conditions? , 1998, International Journal of Computer Vision.

[39]  Rama Chellappa,et al.  Estimation of illuminant direction, albedo, and shape from shading , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Remco C. Veltkamp,et al.  A Survey of 3D Face Recognition Methods , 2005, AVBPA.

[41]  Rama Chellappa,et al.  Symmetric Shape-from-Shading Using Self-ratio Image , 2001, International Journal of Computer Vision.

[42]  Josef Kittler,et al.  Robust albedo estimation from face image under unknown illumination , 2008, SPIE Defense + Commercial Sensing.