Colour invariants under a non-linear photometric camera model and their application to face recognition from video

Illumination invariance remains one of the most researched, yet the most challenging aspect of automatic face recognition. In this paper the discriminative power of colour-based invariants is investigated in the presence of large illumination changes between training and query data, when appearance changes due to cast shadows and non-Lambertian effects are significant. Specifically, there are three main contributions: (i) a general photometric model of the camera is described and it is shown how its parameters can be estimated from realistic video input of pseudo-random head motion, (ii) several novel colour-based face invariants are derived for different special instances of the camera model, and (iii) the performance of the largest number of colour-based representations in the literature is evaluated and analysed on a database of 700 video sequences. The reported results suggest that: (i) colour invariants do have a substantial discriminative power which may increase the robustness and accuracy of recognition from low resolution images in extreme illuminations, and (ii) that the non-linearities of the general photometric camera model have a significant effect on recognition performance. This highlights the limitations of previous work and emphasizes the need to assess face recognition performance using training and query data which had been captured by different acquisition equipment.

[1]  Chengjun Liu,et al.  A Hybrid Color and Frequency Features Method for Face Recognition , 2008, IEEE Transactions on Image Processing.

[2]  Lior Shamir,et al.  Evaluation of Face Datasets as Tools for Assessing the Performance of Face Recognition Methods , 2008, International Journal of Computer Vision.

[3]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[4]  Ioannis T. Pavlidis,et al.  Fusion of Infrared and Visible Images for Face Recognition , 2004, ECCV.

[5]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Andrew Zisserman,et al.  Automatic face recognition for film character retrieval in feature-length films , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[7]  Roberto Cipolla,et al.  Face Recognition from Video Using the Generic Shape-Illumination Manifold , 2006, ECCV.

[8]  Hany Farid,et al.  Blind inverse gamma correction , 2001, IEEE Trans. Image Process..

[9]  Andrew Blake,et al.  Geometry From Specularities , 1988, [1988 Proceedings] Second International Conference on Computer Vision.

[10]  Osamu Nakamura,et al.  Identification of human faces based on isodensity maps , 1991, Pattern Recognit..

[11]  Michael David Kelly,et al.  Visual identification of people by computer , 1970 .

[12]  Roberto Cipolla,et al.  Thermal and reflectance based personal identification methodology under variable illumination , 2010, Pattern Recognit..

[13]  Trevor Darrell,et al.  Face recognition with image sets using manifold density divergence , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  P. Sinha,et al.  Contribution of Color to Face Recognition , 2002, Perception.

[15]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Amnon Shashua,et al.  The Quotient Image: Class-Based Re-Rendering and Recognition with Varying Illuminations , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Rolf P. Würtz,et al.  Elastic Bunch Graph Matching , 2014, Scholarpedia.

[18]  Liming Chen,et al.  Mobile face detection and tracking for media streaming applications , 2007, Int. J. Wirel. Mob. Comput..

[19]  Alice J. O'Toole,et al.  Face Recognition Vendor Test 2006 and Iris Challenge Evaluation 2006 Large-Scale Results | NIST , 2007 .

[20]  Masashi Nishiyama,et al.  Face recognition using the classified appearance-based quotient image , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[21]  M. Tarr,et al.  Gender Recognition of Human Faces Using Color , 2008, Psychological science.

[22]  Raphaël Féraud,et al.  A Fast and Accurate Face Detector Based on Neural Networks , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Yu-Jin Zhang,et al.  State-of-the-Art on Video-Based Face Recognition , 2009, Encyclopedia of Artificial Intelligence.

[24]  Chengjun Liu,et al.  Extracting Multiple Features in the CID Color Space for Face Recognition , 2010, IEEE Transactions on Image Processing.

[25]  Björn Stenger,et al.  Filtering using a tree-based estimator , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[26]  Tae-Kyun Kim,et al.  Boosted manifold principal angles for image set-based recognition , 2007, Pattern Recognit..

[27]  D. Perrett,et al.  Color sensitivity of cells responsive to complex stimuli in the temporal cortex. , 2003, Journal of neurophysiology.

[28]  Donald P. Greenberg,et al.  Color spaces for computer graphics , 1978, SIGGRAPH.

[29]  R. Cipolla,et al.  Thermal and Reflectance Based Personal Identification Methodology in Challenging Variable Illuminations , 2007 .

[30]  Anil K. Jain,et al.  Face Detection in Color Images , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Horst Bischof,et al.  3D and Infrared Face Reconstruction from RGB data using Canonical Correlation Analysis , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[32]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Paola Campadelli,et al.  A Face Recognition System Based on Local Feature Analysis , 2003, AVBPA.

[34]  Takeo Kanade,et al.  Picture Processing System by Computer Complex and Recognition of Human Faces , 1974 .

[35]  Guy Godin,et al.  Separation of diffuse and specular components of surface reflection by use of polarization and statistical analysis of images , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Tat-Jun Chin,et al.  Face Recognition From Video using Active Appearance Model Segmentation , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[37]  Thomas Vetter,et al.  A morphable model for the synthesis of 3D faces , 1999, SIGGRAPH.

[38]  Takeo Kanade,et al.  Surface Reflection: Physical and Geometrical Perspectives , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Marian Stewart Bartlett,et al.  Face recognition by independent component analysis , 2002, IEEE Trans. Neural Networks.

[41]  Payam Saisan,et al.  Dynamic texture recognition , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[42]  Lei Zhang,et al.  Face recognition from a single training image under arbitrary unknown lighting using spherical harmonics , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Pawan Sinha,et al.  Role of color in face recognition , 2010 .

[44]  Sami Romdhani,et al.  Efficient, robust and accurate fitting of a 3D morphable model , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[45]  Caifeng Shan,et al.  Face Recognition and Retrieval in Video , 2010, Video Search and Mining.

[46]  Gene H. Golub,et al.  Numerical methods for computing angles between linear subspaces , 1971, Milestones in Matrix Computation.

[47]  Chengjun Liu,et al.  Color Image Discriminant Models and Algorithms for Face Recognition , 2008, IEEE Transactions on Neural Networks.

[48]  Luis Torres,et al.  The importance of the color information in face recognition , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[49]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[50]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[51]  E. Granum,et al.  Skin colour detection under changing lighting conditions , 1999 .