Detecting skin in face recognition systems: A colour spaces study

Skin colour detection is a technique very used in most of face detectors to find faces in images or videos. However, there is not a common opinion about which colour space is the best choice to do this task. Therefore, the motivation for our study is to discover which colour model is the best option to build an efficient face detector which can be embedded in a functional face recognition system. We have studied 10 of the most common and used colour spaces doing different comparisons among them, in order to know which one is the best option for human skin colour detection. In concrete, we have studied the models: RGB, CMY, YUV, YIQ, YPbPr, YCbCr, YCgCr, YDbDr, HSV-or HSI-and CIE-XYZ. To make the comparison among them, we have used 15 truth images where the skin colour of a face is clearly separated from the rest of the image (background, eyes, lips, hair, etc.). Thus we can compare at level pixel each colour model, doing a detailed study of each format. We present the final conclusions comparing different results, such as: right detections, false positives and false negatives for each colour space. According to the obtained results, the most appropriate colour spaces for skin colour detection are HSV model (the winner in our study), and the models YCgCr and YDbDr.

[1]  Farid Boussaïd,et al.  On-chip skin detection for color CMOS imagers , 2003, Proceedings International Conference on MEMS, NANO and Smart Systems.

[2]  Heitor S. Ramos,et al.  Skin Detection in Web Imagery: Comparison of Techniques and Proposal , 2006, 2006 International Conference on Image Processing.

[3]  King Ngi Ngan,et al.  Face segmentation using skin-color map in videophone applications , 1999, IEEE Trans. Circuits Syst. Video Technol..

[4]  Tao Zhang,et al.  Real-time Vision-based Gesture Recognition for Human Robot Interaction , 2004, 2004 IEEE International Conference on Robotics and Biomimetics.

[5]  King Ngi Ngan,et al.  Locating facial region of a head-and-shoulders color image , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[6]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Azriel Rosenfeld,et al.  Computer Vision , 1988, Adv. Comput..

[8]  J. Tasic,et al.  Colour spaces: perceptual, historical and applicational background , 2003, The IEEE Region 8 EUROCON 2003. Computer as a Tool..

[9]  Koji Hashimoto,et al.  Facial expression recognition of person without language ability based on the optical flow histogram , 2000, WCC 2000 - ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000.

[10]  T. Yamada,et al.  Analysis and synthesis of facial color for the affect display of virtual facial image under fearful emotion , 2005, Proceedings of the 2005 International Conference on Active Media Technology, 2005. (AMT 2005)..

[11]  A. Bouzerdoum,et al.  A Bayesian approach to skin color classification in YCbCr color space , 2000, 2000 TENCON Proceedings. Intelligent Systems and Technologies for the New Millennium (Cat. No.00CH37119).

[12]  Yves Vander Haeghen,et al.  An imaging system with calibrated color image acquisition for use in dermatology , 2000, IEEE Transactions on Medical Imaging.

[13]  Rama Chellappa,et al.  Image-Based Face Recognition: Issues and Methods , 2002 .

[14]  Chang-Tsun Li,et al.  Skin Colour-Based Face Detection in Colour Images , 2006, 2006 IEEE International Conference on Video and Signal Based Surveillance.

[15]  Ying Dai,et al.  Face-texture model based on SGLD and its application in face detection in a color scene , 1996, Pattern Recognit..

[16]  Stan Sclaroff,et al.  Estimation and prediction of evolving color distributions for skin segmentation under varying illumination , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[17]  Jason Brand,et al.  A comparative assessment of three approaches to pixel-level human skin-detection , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[18]  Jun Gao,et al.  Color Edge Detection Based on YUV Space and Minimal Spanning Tree , 2006, 2006 IEEE International Conference on Information Acquisition.

[19]  Shaogang Gong,et al.  Tracking colour objects using adaptive mixture models , 1999, Image Vis. Comput..

[20]  A. Chekima,et al.  Lips Detection in the Normalised RGB Colour Scheme , 2006, 2006 2nd International Conference on Information & Communication Technologies.

[21]  David C. Gibbon,et al.  Multi-modal system for locating heads and faces , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[22]  S. L. Phung,et al.  A novel skin color model in YCbCr color space and its application to human face detection , 2002, Proceedings. International Conference on Image Processing.

[23]  He Kun,et al.  Face detection in color images based on sub-image fusion and SVM , 2005, IEEE International Conference Mechatronics and Automation, 2005.

[24]  Wen-Ke Wang,et al.  Extraction of face regions in color image , 2002, Proceedings. International Conference on Machine Learning and Cybernetics.

[25]  M. Fukumi,et al.  Feature extraction for face detection and recognition , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[26]  Abdesselam Bouzerdoum,et al.  Skin segmentation using color pixel classification: analysis and comparison , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  R. Naguib,et al.  Hand motion segmentation against skin colour background in breast awareness applications , 2004, The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[28]  Mohammad Al-Amin Bhuiyan,et al.  On Gesture Recognition for Human-Robot Symbiosis , 2006, ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication.

[29]  Elena Casiraghi,et al.  A Face Detection System Based on Color and Support Vector Machines , 2003, WIRN.

[30]  Vijayan K. Asari,et al.  Neural network based skin color model for face detection , 2003, 32nd Applied Imagery Pattern Recognition Workshop, 2003. Proceedings..

[31]  Zhan Yong-zhao.  Research and Implementation of a Real-time Approach to Lip Detection in Video Sequences , 2004 .

[32]  Ming-Chieh Chi,et al.  ROI video coding based on H.263+ with robust skin-color detection technique , 2003, IEEE Trans. Consumer Electron..

[33]  Paola Campadelli,et al.  Face and Facial Feature Localization , 2005, ICIAP.

[34]  William K. Pratt,et al.  Digital Image Processing: PIKS Inside , 2001 .

[35]  S. Aramvith,et al.  Face and hands localization and tracking for sign language recognition , 2004, IEEE International Symposium on Communications and Information Technology, 2004. ISCIT 2004..

[36]  Linda G. Shapiro,et al.  Computer Vision , 2001 .

[37]  Vijayan K. Asari,et al.  Associative memory based on ratio learning for real time skin color detection , 2003, 32nd Applied Imagery Pattern Recognition Workshop, 2003. Proceedings..

[38]  Kyung-Yung Choi,et al.  Facial feature extraction from a video sequence using independent component analysis (ICA) , 2001, 2001 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (IEEE Cat. No.01CH37233).

[39]  Somchai Jitapunkul,et al.  Face segmentation based on Hue-Cr components and morphological technique , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[40]  Abdesselam Bouzerdoum,et al.  A current-mode VLSI architecture for skin detection , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[41]  Stan Sclaroff,et al.  Skin color-based video segmentation under time-varying illumination , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Giovani Gómez.  On selecting colour components for skin detection , 2002, Object recognition supported by user interaction for service robots.

[43]  Min C. Shin,et al.  Effect of colorspace transformation, the illuminance component, and color modeling on skin detection , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[44]  Osamu Ikeda,et al.  Segmentation of faces in video footage using HSV color for face detection and image retrieval , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[45]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[46]  Euntai Kim,et al.  TSK Fuzzy modeling approach for Face detection , 2006, 2006 SICE-ICASE International Joint Conference.

[47]  Alex Pentland,et al.  Parametrized structure from motion for 3D adaptive feedback tracking of faces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[48]  Hideyuki Ebine,et al.  Extraction of faces of more than one person from natural background for personal identification , 2001, Canadian Conference on Electrical and Computer Engineering 2001. Conference Proceedings (Cat. No.01TH8555).

[49]  K. Jo,et al.  Multi-face Detection System in Video Sequence , 2006, 2006 International Forum on Strategic Technology.

[50]  You Zhi-sheng,et al.  Face detection and facial feature extraction in color image , 2003, Proceedings Fifth International Conference on Computational Intelligence and Multimedia Applications. ICCIMA 2003.

[51]  Tieniu Tan,et al.  Adaptive skin detection using multiple cues , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[52]  José Santos-Victor,et al.  Active face and feature tracking , 1999, Proceedings 10th International Conference on Image Analysis and Processing.

[53]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[54]  Masafumi Hagiwara,et al.  Color-illustration generation system from facial image , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[55]  Narciso García,et al.  Face detection based on a new color space YCgCr , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[56]  Rita Cucchiara,et al.  Color calibration for a dermatological video camera system , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[57]  Masanori Yamada,et al.  A new robust real-time method for extracting human silhouettes from color images , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[58]  M. Fukumi,et al.  Face recognition using genetic algorithm based template matching , 2004, IEEE International Symposium on Communications and Information Technology, 2004. ISCIT 2004..

[59]  John R. Kender,et al.  Finding skin in color images , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[60]  Jaihie Kim,et al.  Adaptive skin detection system on DTV , 2004, Proceedings of 2004 International Symposium on Intelligent Signal Processing and Communication Systems, 2004. ISPACS 2004..

[61]  S. Hernández,et al.  Face detection by polling , 2005, IEEE International Workshop on Intelligent Signal Processing, 2005..

[62]  Naohiro Ishii,et al.  Extraction of face region by using characteristics of color space and detection of face direction through an eigenspace , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[63]  Ioannis Pitas,et al.  Face localization and facial feature extraction based on shape and color information , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[64]  Oliver Schreer,et al.  Fast and robust shadow detection in videoconference applications , 2002, International Symposium on VIPromCom Video/Image Processing and Multimedia Communications.

[65]  Peter J. Wolfs,et al.  Robustness of colour detection for robot soccer , 2002, 7th International Conference on Control, Automation, Robotics and Vision, 2002. ICARCV 2002..

[66]  Mohamed A. Deriche,et al.  Robust human face detection in complex color images , 2005, IEEE International Conference on Image Processing 2005.

[67]  Alex Pentland,et al.  Mixtures of eigenfeatures for real-time structure from texture , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[68]  Narciso García,et al.  Fast face segmentation in component color space , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[69]  Ying Liu,et al.  Study on digital imaging colorimeter in dermatology , 2004, The Second Asian and Pacific Rim Symposium on Biophotonics, 2004. APBP 2004..

[70]  Orlando J. Hernandez,et al.  Face recognition using multispectral random field texture models, color content, and biometric features , 2005, 34th Applied Imagery and Pattern Recognition Workshop (AIPR'05).

[71]  Edward J. Delp,et al.  Optimum color spaces for skin detection , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[72]  R. Jailani,et al.  Color space for psoriasis skin diseases analysis , 2003, Asian Conference on Sensors, 2003. AsiaSense 2003..

[73]  Alexander H. Waibel,et al.  Visual tracking for multimodal human computer interaction , 1998, CHI.

[74]  Abdesselam Bouzerdoum,et al.  A Bayesian skin/non-skin color classifier using non-parametric density estimation , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[75]  Y. Miyake,et al.  Facial pattern detection and color correction from television picture for newspaper printing , 1990 .

[76]  Chiunhsiun Lin,et al.  Face detection by color and multilayer feedforward neural network , 2005, 2005 IEEE International Conference on Information Acquisition.

[77]  Kang-Hyun Jo,et al.  Color-based Face Detection using Combination of Modified Local Binary Patterns and embedded Hidden Markov Models , 2006, 2006 SICE-ICASE International Joint Conference.

[78]  Weimin Huang,et al.  Face detection based on color and local symmetry information , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.

[79]  Iwate Takizawa.  A NEW METHOD OF FACIAL ACTION EXTRACTION AND EXPRESSION RECOGNITION OF PERSON WITHOUT LANGUAGE Z Dai Z Shibata K. Hashimoto D. Cai K. Katamachi T. Ishii A. Osuo and K. Nokuchi , 2000 .

[80]  Alexander H. Waibel,et al.  A real-time face tracker , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[81]  Li-Qun Xu.  Simultaneous tracking and segmentation of two free moving hands in a video conferencing scenario , 2003 .

[82]  Seok Cheol Kee,et al.  Speaker detection and tracking at mobile robot platform , 2004, Proceedings of 2004 International Symposium on Intelligent Signal Processing and Communication Systems, 2004. ISPACS 2004..

[83]  Jason M. Kinser,et al.  Optimizing image segmentation using color model mixtures , 2005, 34th Applied Imagery and Pattern Recognition Workshop (AIPR'05).

[84]  Takeo Kanade,et al.  Name-It: Naming and Detecting Faces in News Videos , 1999, IEEE Multim..

[85]  Georgios Tziritas,et al.  Face Detection Using Quantized Skin Color Regions Merging and Wavelet Packet Analysis , 1999, IEEE Trans. Multim..

[86]  Christos Grecos,et al.  A fast skin region detector for colour images , 2005 .

[87]  Cheng-Chew Lim,et al.  Automatic human skin segmentation based on color information in the YCbCr color space , 2002, Final Program and Abstracts on Information, Decision and Control.

[88]  Y. Takahashi,et al.  Feature Point Extraction in Face Image by Neural Network , 2006, 2006 SICE-ICASE International Joint Conference.

[89]  Vijayan K. Asari,et al.  Face detection technique based on intensity and skin color distribution , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[90]  Noel E. O'Connor,et al.  A hybrid technique for face detection in color images , 2005, IEEE Conference on Advanced Video and Signal Based Surveillance, 2005..

[91]  Ying Dai,et al.  Extraction of Facial Images from the Complex Background Using Color Information and SGLD Matrices , 1996 .

[92]  Li Bo,et al.  A practical algorithm for exception event detection for the home video security surveillance , 2001, 2001 International Conferences on Info-Tech and Info-Net. Proceedings (Cat. No.01EX479).

[93]  Qian Chen,et al.  Face detection by fuzzy pattern matching , 1995, Proceedings of IEEE International Conference on Computer Vision.