Addressing the illumination challenge in two-dimensional face recognition: a survey

Uncontrolled illumination is one of the most widely researched and most encountered face recognition challenges in recent years. In this study, the authors propose the division of algorithms into two categories: (i) relighting and (ii) unlighting. Relighting methods try to match the probe's illumination conditions using a subset of representative gallery images, while unlighting methods seek to suppress the variations. A total of 64 state-of-the-art methods are summarised and categorised in each of the groups. To make this work concise and easy to follow, they restricted themselves to selected conferences/journals and they limited the number of approaches reviewed. Also, eight past state-of-the-art approaches are used in both identification and verification experiments. However, only significant reported results from all methods were compared and organised in tables. The author's main objective is not to provide an exhaustive analysis of each category, but to present a collection of papers that can be useful in identifying research directions. Results indicate that unlighting methods are a better and a practical solution to address illumination challenges.

[1]  Chu-Song Chen,et al.  Intrinsic Illumination Subspace for Lighting Insensitive Face Recognition , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  Xiaohua Xie,et al.  A Study on the Effective Approach to Illumination-Invariant Face Recognition Based on a Single Image , 2012, CCBR.

[3]  Zia-ur Rahman,et al.  Properties and performance of a center/surround retinex , 1997, IEEE Trans. Image Process..

[4]  Soo-Hyung Kim,et al.  De-Noising Model for Weberface-Based and Max-Filter-Based Illumination Invariant Face Recognition , 2014 .

[5]  Meng Joo Er,et al.  A novel efficient local illumination compensation method based on DCT in logarithm domain , 2012, Pattern Recognit. Lett..

[6]  Jian-Huang Lai,et al.  Logarithm Gradient Histogram: A general illumination invariant descriptor for face recognition , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[7]  Jie Yang,et al.  Illumination Processing Recognition of Face Images Based on Improved Retinex Algorithm , 2013, J. Multim..

[8]  Biao Wang,et al.  Illumination Normalization Based on Weber's Law With Application to Face Recognition , 2011, IEEE Signal Processing Letters.

[9]  Q. M. Jonathan Wu,et al.  Illumination invariant human face recognition: frequency or resonance? , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[10]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Wael Badawy,et al.  Eliminating illumination effects by discrete cosine transform (DCT) coefficients' attenuation and accentuation , 2013, Electronic Imaging.

[12]  Song Han,et al.  Local Illumination Normalization and Facial Feature Point Selection for Robust Face Recognition , 2013 .

[13]  Li Lin,et al.  Low cost illumination invariant face recognition by down-up sampling self quotient image , 2013, 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.

[14]  Rahman Zia-ur,et al.  A Comparison of the Multiscale Retinex With Other Image Enhancement Techniques , 1997 .

[15]  Chun-Nian Fan,et al.  Homomorphic filtering based illumination normalization method for face recognition , 2011, Pattern Recognit. Lett..

[16]  Christophe Garcia,et al.  A Wavelet-based Framework for Face Recognition , 1998 .

[17]  Yi Zhou,et al.  A de-illumination scheme for face recognition based on fast decomposition and detail feature fusion. , 2013, Optics express.

[18]  Chunheng Wang,et al.  Sparse representation for face recognition based on discriminative low-rank dictionary learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Jun Miura,et al.  Fuzzy-based illumination normalization for face recognition , 2013, 2013 IEEE Workshop on Advanced Robotics and its Social Impacts.

[20]  Minal Patel,et al.  DWT-based Illumination Normalization and Feature Extraction for Enhanced Face Recognition , 2012 .

[21]  Wen Gao,et al.  The CAS-PEAL Large-Scale Chinese Face Database and Baseline Evaluations , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[22]  Ashish Ghosh,et al.  An efficient illumination invariant face recognition technique using two dimensional linear discriminant analysis , 2012, 2012 1st International Conference on Recent Advances in Information Technology (RAIT).

[23]  Housam Khalifa Bashier,et al.  A novel illumination normalization algorithm for face recognition , 2013, 2013 IEEE International Conference on Signal and Image Processing Applications.

[24]  Baining Guo,et al.  Face Synthesis , 2011, Handbook of Face Recognition.

[25]  David W. Jacobs,et al.  In search of illumination invariants , 2001, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[26]  Enzo Pasquale Scilingo,et al.  Comparitive study on photometric normalization algorithms for an innovative, robust and real-time eye gaze tracker , 2011, Journal of Real-Time Image Processing.

[27]  Yuan Yan Tang,et al.  Illumination Invariant Face Recognition Using Fabemd Decomposition with Detail Measure Weight , 2011, Int. J. Pattern Recognit. Artif. Intell..

[28]  Shree K. Nayar,et al.  Eyes for relighting , 2004, SIGGRAPH 2004.

[29]  Meng Joo Er,et al.  A Novel Face Recognition Approach under Illumination Variations Based on Local Binary Pattern , 2011, CAIP.

[30]  M. Biglari,et al.  Illumination invariant face recognition using SQI and weighted LBP histogram , 2013, 2013 First Iranian Conference on Pattern Recognition and Image Analysis (PRIA).

[31]  Chu-Song Chen,et al.  Lighting normalization with generic intrinsic illumination subspace for face recognition , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[32]  Majid Ahmadi,et al.  An efficient illumination invariant face recognition framework via illumination enhancement and DD-DTCWT filtering , 2013, Pattern Recognit..

[33]  Ehsanollah Kabir,et al.  Visual illumination compensation for face images using light mapping matrix , 2013, IET Image Process..

[34]  Bin Fang,et al.  A Fusion Framework for Face Recognition under Varying Illumination Based on Multi-Scale Analysis , 2012, Int. J. Wavelets Multiresolution Inf. Process..

[35]  Rama Chellappa,et al.  Illumination robust dictionary-based face recognition , 2011, 2011 18th IEEE International Conference on Image Processing.

[36]  Haifeng Hu,et al.  Variable lighting face recognition using discrete wavelet transform , 2011, Pattern Recognit. Lett..

[37]  Javier Ruiz-del-Solar,et al.  Illumination compensation and normalization in eigenspace-based face recognition: A comparative study of different pre-processing approaches , 2008, Pattern Recognit. Lett..

[38]  Ioannis A. Kakadiaris,et al.  Illumination Normalization Using Self-lighting Ratios for 3D2D Face Recognition , 2012, ECCV Workshops.

[39]  Jie Lin,et al.  Robust face recognition with partial occlusion, illumination variation and limited training data by optimal feature selection , 2011 .

[40]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Sang-Heon Lee,et al.  Illumination-robust face recognition system based on differential components , 2012, IEEE Transactions on Consumer Electronics.

[42]  Dong-Ju Kim,et al.  Illumination-Robust Local Pattern Descriptor for Face Recognition , 2014 .

[43]  Sanun Srisuk Robust Face Recognition Based on Texture Analysis , 2013 .

[44]  P. McOwan,et al.  Illumination robust face representation based on intrinsic geometrical information , 2012 .

[45]  V. P. Vishwakarma,et al.  Rescaling of low frequency DCT coefficients with Kernel PCA for illumination invariant face recognition , 2013, 2013 3rd IEEE International Advance Computing Conference (IACC).

[46]  Minghua Zhao,et al.  The discrete cosine transform (DCT) plus local normalization: a novel two-stage method for de-illumination in face recognition , 2011 .

[47]  Conrad Sanderson,et al.  On Local Features for Face Verification , 2004 .

[48]  Thomas B. Moeslund,et al.  Special issue on Multimedia Event Detection , 2013, Machine Vision and Applications.

[49]  Amnon Shashua,et al.  The Quotient Image: Class-Based Re-Rendering and Recognition with Varying Illuminations , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  Meng Joo Er,et al.  A Novel Local Illumination Normalization Approach for Face Recognition , 2011, ISNN.

[51]  Chi-Ho Chan,et al.  Photometric Normalization for Face Recognition using Local discrete cosine Transform , 2013, Int. J. Pattern Recognit. Artif. Intell..

[52]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[53]  Yong Cheng,et al.  Illumination Normalization for Face Recognition under Extreme Lighting Conditions , 2012, IScIDE.

[54]  Yutaka Satoh,et al.  Robust face recognition using the GAP feature , 2013, Pattern Recognit..

[55]  Heydi Mendez Vazquez,et al.  Illumination Invariant Face Recognition Using Quaternion-Based Correlation Filters , 2012, Journal of Mathematical Imaging and Vision.

[56]  Hui Wang,et al.  Face recognition under varying illumination , 2012, Neural Computing and Applications.

[57]  M Xie,et al.  Intelligent robotics and applications (2nd international conference ICIRA 2009) , 2009 .

[58]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[59]  Yong Luo,et al.  A Robust Illumination Normalization Method Based on Mean Estimation for Face Recognition , 2013 .

[60]  Meng Joo Er,et al.  Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[61]  Xiaoyang Tan,et al.  Enhanced Local Texture Feature Sets for Face Recognition Under Difficult Lighting Conditions , 2007, IEEE Transactions on Image Processing.

[62]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Jian-Huang Lai,et al.  Non-ideal class non-point light source quotient image for face relighting , 2011, Signal Process..

[64]  Sinjini Mitra,et al.  Gaussian Mixture Models for Human Face Recognition under Illumination Variations , 2012 .

[65]  Rynson W. H. Lau,et al.  Multimedia and Signal Processing , 2012, Communications in Computer and Information Science.

[66]  Mao Ye,et al.  Shadow compensation and illumination normalization of face image , 2013, Machine Vision and Applications.

[67]  Xue Yuan,et al.  Illumination Normalization Based on Homomorphic Wavelet Filtering for Face Recognition , 2013, J. Inf. Sci. Eng..

[68]  Anil K. Jain,et al.  Open source biometric recognition , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[69]  Jing Wang,et al.  Scalable illumination robust face identification using harmonic representation , 2013, Other Conferences.

[70]  Yuanyuan Ma,et al.  Illumination Invariant Face Recognition Based on Nonsubsampled Contourlet Transform and NeighShrink Denoise , 2012, MMSP 2012.

[71]  Saeid Pashazadeh,et al.  Robust Recognition against Illumination Variations Based on SIFT , 2012, ICIRA.

[72]  Ramji M. Makwana Illumination invariant face recognition: A survey of passive methods , 2010, Biometrics Technology.

[73]  Gregory Shakhnarovich,et al.  Face Recognition in Subspaces , 2011, Handbook of Face Recognition.

[74]  Alice Caplier,et al.  Illumination-robust face recognition using retina modeling , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[75]  Si-Yu Xia,et al.  Illumination invariant face recognition based on improved Local Binary Pattern , 2011, Proceedings of the 30th Chinese Control Conference.

[76]  Tatsuo Arai,et al.  Face relighting using discriminative 2D spherical spaces for face recognition , 2013, Machine Vision and Applications.

[77]  Aly A. Farag,et al.  Towards efficient image irradiance modelling of convex Lambertian surfaces under single viewpoint and frontal illumination , 2013, IET Comput. Vis..

[78]  Aly A. Farag,et al.  Analytic Bilinear Appearance Subspace Construction for Modeling Image Irradiance under Natural Illumination and Non-Lambertian Reflectance , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[79]  Marko Heikkilä,et al.  Description of interest regions with local binary patterns , 2009, Pattern Recognit..

[80]  Nitin Kumar,et al.  Performance evaluation of linear subspace methods for face recognition under illumination variation , 2011, C3S2E '11.

[81]  Vitomir Struc,et al.  Photometric Normalization Techniques for Illumination Invariance , 2011 .

[82]  Xiaohua Xie Illumination preprocessing for face images based on empirical mode decomposition , 2014, Signal Process..

[83]  Pritee Khanna,et al.  Face verification and identification using DCT-NNDA and SIFT with score-level fusion , 2013 .

[84]  Heydi Mendez Vazquez,et al.  Illumination Invariant Face Recognition in quaternion Domain , 2013, Int. J. Pattern Recognit. Artif. Intell..

[85]  Aly A. Farag,et al.  Image irradiance harmonics: a phenomenological model of image irradiance of arbitrary surface reflectance , 2014, IET Comput. Vis..

[86]  P. K. Banerjee,et al.  Phase-eigen subspace based illumination invariant face recognition using associative memory , 2012, 2012 NATIONAL CONFERENCE ON COMPUTING AND COMMUNICATION SYSTEMS.

[87]  Jing-Yu Yang,et al.  Illumination invariant extraction for face recognition using neighboring wavelet coefficients , 2012, Pattern Recognit..

[88]  Baoxin Li,et al.  Discriminative K-SVD for dictionary learning in face recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[89]  Josef Kittler,et al.  A comparison of photometric normalisation algorithms for face verification , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[90]  Chu-Song Chen,et al.  Illumination invariant feature extraction based on natural images statistics — Taking face images as an example , 2011, CVPR 2011.

[91]  Gee-Sern Hsu,et al.  Face recognition using sparse representation with illumination normalization and component features , 2013, 2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference.

[92]  Haitao Wang,et al.  Face recognition under varying lighting conditions using self quotient image , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[93]  Yuan Yan Tang,et al.  Illumination invariant face recognition based on the new phase features , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[94]  Wen Gao,et al.  A comparative study on illumination preprocessing in face recognition , 2013, Pattern Recognit..

[95]  Hong Yan,et al.  Wavelets and Face Recognition , 2007 .

[96]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[97]  Hyeonjoon Moon,et al.  The FERET Evaluation Methodology for Face-Recognition Algorithms , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[98]  Hujun Yin,et al.  Eigenlights: Recovering Illumination from Face Images , 2011, IDEAL.

[99]  Songbai Pu,et al.  Detail-Enhance Face Illumination Normalization Based on LDCT-Wavelet , 2013 .

[100]  Yong Gao Jin,et al.  Research on Importance of Texture Information in Face Recognition , 2013 .

[101]  Mukesh A. Zaveri,et al.  Rough membership function based illumination classifier for illumination invariant face recognition , 2013, Appl. Soft Comput..

[102]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[103]  Rabab Kreidieh Ward,et al.  Wavelet-based illumination normalization for face recognition , 2005, IEEE International Conference on Image Processing 2005.

[104]  Xuan Zou,et al.  Illumination Invariant Face Recognition: A Survey , 2007, 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems.

[105]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[106]  Chi Fang,et al.  Generating face images under multiple illuminations based on a single front-lighted sample without 3D models , 2013, 2013 International Conference on Biometrics (ICB).

[107]  Yi Li,et al.  Shadow determination and compensation for face recognition , 2013, International Journal of Machine Learning and Cybernetics.

[108]  Biao Wang,et al.  Illumination Variation Dictionary Designing for Single-Sample Face Recognition via Sparse Representation , 2013, MMM.

[109]  Xi Chen,et al.  Illumination robust single sample face recognition using multi-directional orthogonal gradient phase faces , 2011, Neurocomputing.

[110]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[111]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[112]  Virendra P. Vishwakarma,et al.  Illumination normalization using down-scaling of low-frequency DCT coefficients in DWT domain for face recognition , 2013, 2013 Sixth International Conference on Contemporary Computing (IC3).

[113]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[114]  Dao-Qing Dai,et al.  Face Recognition under Variable Illumination by Weighted-Subband Edge Enhancement , 2012, CCPR.

[115]  Virendra P. Vishwakarma,et al.  Illumination normalization using fuzzy filter in DCT domain for face recognition , 2013, International Journal of Machine Learning and Cybernetics.