A comparative study of preprocessing mismatch effects in color image based face recognition

Face color information can play an important role in face recognition (FR) and it can be used to considerably improve FR performance obtained using only grayscale images. The color-based FR methods involve a preprocessing step where a color image is converted into either a monochromatic image or an image having a different color representation. In practical FR systems, the recording or transmission format of the testing images may be arbitrary or inconsistent depending on the application (e.g., face images could consist of grayscale or color pixels either in compressed or uncompressed form). Further, a wide variety of grayscale and color conversions can be used in the preprocessing step. This could lead to a so-called preprocessing mismatch in color-based FR methods: the training and testing face images, generated after preprocessing, do not match in terms of their degree of compression or in terms of their grayscale or color representations. In contrast to grayscale-based FR, a practical color-based FR system has a higher chance of being confronted with a preprocessing mismatch. The aim of this paper is to present a comparative study that addresses the impact of a preprocessing mismatch on color-image based FR methods. We explore three different types of preprocessing mismatches, which practical color-based FR system are highly likely to encounter. In addition, comparative and extensive experiments have been carried out to analyze the effects of the preprocessing mismatches on an FR performance, using Color FRETET, CMU-PIE, AR, and SCface public face databases. The important conclusions drawn from our experiments include: (1) of all color-based FR methods under consideration, color-based FR using feature-level fusion is the most robust approach to preprocessing mismatches; (2) the preprocessing mismatch caused by the use of compressed color images can significantly deteriorate FR performance of color-based FR methods; (3) grayscale testing images can be critical for the feasibility of color-based FR using an input-level fusion; (4) the preprocessing mismatch in terms of grayscale representation has little effect on the FR performances of color-based FR methods.

[1]  Yong Man Ro,et al.  Face annotation for personal photos using collaborative face recognition in online social networks , 2009, 2009 16th International Conference on Digital Signal Processing.

[2]  Alex Pentland,et al.  Bayesian face recognition , 2000, Pattern Recognit..

[3]  Konstantinos N. Plataniotis,et al.  Face recognition using kernel direct discriminant analysis algorithms , 2003, IEEE Trans. Neural Networks.

[4]  Yong Man Ro,et al.  Color Face Recognition for Degraded Face Images , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Gregory K. Wallace,et al.  The JPEG still picture compression standard , 1991, CACM.

[7]  Amin Milani Fard,et al.  Cooperative criminal face recognition in distributed web environment , 2008, 2008 IEEE/ACS International Conference on Computer Systems and Applications.

[8]  A. Lynn Abbott,et al.  EURASIP Journal on Applied Signal Processing 2004:4, 1–8 c ○ 2004 Hindawi Publishing Corporation Optimization of Color Conversion for Face Recognition , 2003 .

[9]  Christian Schneider,et al.  Feature based Face Localization and Recognition on Mobile Devices , 2006, 2006 9th International Conference on Control, Automation, Robotics and Vision.

[10]  Mohan S. Kankanhalli,et al.  Application Potential of Multimedia Information Retrieval , 2008, Proceedings of the IEEE.

[11]  David Zhang,et al.  A Multiple Maximum Scatter Difference Discriminant Criterion for Facial Feature Extraction , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Pawan Sinha,et al.  Role of color in face recognition , 2010 .

[14]  Josef Kittler,et al.  Physics-Based Decorrelation of Image Data for Decision Level Fusion in Face Verification , 2004, Multiple Classifier Systems.

[15]  Ramesh C. Jain,et al.  Multimedia information retrieval: watershed events , 2008, MIR '08.

[16]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Josef Kittler,et al.  Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Josef Kittler,et al.  Confidence Based Gating of Colour Features for Face Authentication , 2007, MCS.

[19]  Chengjun Liu,et al.  Comparative assessment of content-based face image retrieval in different color spaces , 2005, Int. J. Pattern Recognit. Artif. Intell..

[20]  K. Plataniotis,et al.  Color Image Processing : Methods and Applications , 2006 .

[21]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[22]  Chengjun Liu,et al.  Learning the Uncorrelated, Independent, and Discriminating Color Spaces for Face Recognition , 2008, IEEE Transactions on Information Forensics and Security.

[23]  Trevor Darrell,et al.  Autotagging Facebook: Social network context improves photo annotation , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[24]  Shibnath Mukherjee Deloitte.  A Secure Face Recognition System for Mobile-devices without The Need of Decryption , 2008 .

[25]  Chengjun Liu,et al.  Improving the Face Recognition Grand Challenge Baseline Performance using Color Configurations Across Color Spaces , 2006, 2006 International Conference on Image Processing.

[26]  Qiang Ji,et al.  A Comparative Study of Local Matching Approach for Face Recognition , 2007, IEEE Transactions on Image Processing.

[27]  Kwontaeg Choi,et al.  A collaborative face recognition framework on a social network platform , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[28]  Ming Zhao,et al.  Large scale learning and recognition of faces in web videos , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[29]  Werner von Seelen,et al.  Integrating Face Recognition into Security Systems , 1997, AVBPA.

[30]  Theo Gevers,et al.  Selection and Fusion of Color Models for Image Feature Detection , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  P. Jonathon Phillips,et al.  Face Recognition Vendor Test 2002 Performance Metrics , 2003, AVBPA.

[32]  Tejaswini Ganapathi.  Color Image Based Face Recognition , 2009 .

[33]  Brian C. Lovell,et al.  Experimental Analysis of Face Recognition on Still and CCTV Images , 2008, 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance.

[34]  Konstantinos N. Plataniotis,et al.  Regularized discriminant analysis for the small sample size problem in face recognition , 2003, Pattern Recognit. Lett..

[35]  Shihong Lao,et al.  Security Management for Mobile Devices by Face Recognition , 2006, 7th International Conference on Mobile Data Management (MDM'06).

[36]  MandalBappaditya,et al.  Eigenfeature Regularization and Extraction in Face Recognition , 2008 .

[37]  Alan F. Smeaton,et al.  Context-Aware Person Identification in Personal Photo Collections , 2009, IEEE Transactions on Multimedia.

[38]  Cheng-Lin Liu,et al.  Classifier combination based on confidence transformation , 2005, Pattern Recognit..

[39]  Aleix M. Martinez,et al.  The AR face database , 1998 .

[40]  Jie Wang,et al.  On solving the face recognition problem with one training sample per subject , 2006, Pattern Recognit..

[41]  Jay Yagnik,et al.  Learning people annotation from the web via consistency learning , 2007, MIR '07.

[42]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[43]  Luis Torres,et al.  Automatic face recognition for video indexing applications , 2002, Pattern Recognit..

[44]  A. Martínez,et al.  The AR face databasae , 1998 .

[45]  Wen Gao,et al.  Hierarchical Ensemble of Global and Local Classifiers for Face Recognition , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[46]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Chengjun Liu,et al.  A Hybrid Color and Frequency Features Method for Face Recognition , 2008, IEEE Transactions on Image Processing.

[48]  Yong Man Ro,et al.  Face annotation for personal photos using context-assisted face recognition , 2008, MIR '08.

[49]  Yong Man Ro,et al.  Feature subspace determination in video-based mismatched face recognition , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[50]  Michael G. Strintzis,et al.  Use of depth and colour eigenfaces for face recognition , 2003, Pattern Recognit. Lett..

[51]  Josef Kittler,et al.  A comparative study of data fusion strategies in face verification , 2004, 2004 12th European Signal Processing Conference.

[52]  Jagath C. Rajapakse,et al.  Color channel encoding with NMF for face recognition , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[53]  Baback Moghaddam,et al.  Principal Manifolds and Probabilistic Subspaces for Visual Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Chengjun Liu,et al.  Comparative Performance Evaluation of Gray-Scale and Color Information for Face Recognition Tasks , 2001, AVBPA.

[55]  Arun Ross,et al.  Score normalization in multimodal biometric systems , 2005, Pattern Recognit..

[56]  Chengjun Liu,et al.  Color Image Discriminant Models and Algorithms for Face Recognition , 2008, IEEE Transactions on Neural Networks.

[57]  Luis Torres,et al.  The importance of the color information in face recognition , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[58]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[59]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[60]  Chengjun Liu,et al.  Color space normalization: Enhancing the discriminating power of color spaces for face recognition , 2010, Pattern Recognit..

[61]  Mislav Grgic,et al.  SCface – surveillance cameras face database , 2011, Multimedia Tools and Applications.