Analysing Facial Features Using CNNs and Computer Vision

This paper presents an automatic facial analysis system which is able to perform gender detection, hair segmentation and geometry detection, color attributes extraction (hair, skin, eyebrows, eyes and lips), accessories (eyeglasses) analysis from facial images. For the more complex tasks (gender detection, hair segmentation, eyeglasses detection) we used state of the art convolutional neural networks, and for the other tasks we used classical image processing algorithms based on geometry and appearance models. When data was available, the proposed system was evaluated on public datasets. An acceptance study was also performed to assess the performance on the system in real life scenarios.

[1]  Radu Danescu,et al.  In the Eye of the Deceiver: Analyzing Eye Movements as a Cue to Deception , 2018, J. Imaging.

[2]  Pierre-Yves Coulon,et al.  Frequential and color analysis for hair mask segmentation , 2008, 2008 15th IEEE International Conference on Image Processing.

[3]  Tal Hassner,et al.  Age and Gender Estimation of Unfiltered Faces , 2014, IEEE Transactions on Information Forensics and Security.

[4]  K. Bowyer,et al.  The Importance of Small Pupils: A Study of How Pupil Dilation Affects Iris Biometrics , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[5]  Joshua Correll,et al.  The Chicago face database: A free stimulus set of faces and norming data , 2015, Behavior research methods.

[6]  Yongzhao Zhan,et al.  Maximum Neighborhood Margin Discriminant Projection for Classification , 2014, TheScientificWorldJournal.

[7]  Rabah Attia,et al.  Classification of Human Skin Color and its Application to Face Recognition , 2014, MMEDIA 2014.

[8]  Jean-Luc Dugelay,et al.  On the reliability of eye color as a soft biometric trait , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[9]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[11]  Qiang Ji,et al.  In the Eye of the Beholder: A Survey of Models for Eyes and Gaze , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[13]  Sabine Süsstrunk,et al.  Consistent image-based measurement and classification of skin color , 2005, IEEE International Conference on Image Processing 2005.

[14]  Riccardo Leonardi,et al.  Hair detection, segmentation, and hairstyle classification in the wild , 2018, Image Vis. Comput..

[15]  Davis E. King,et al.  Dlib-ml: A Machine Learning Toolkit , 2009, J. Mach. Learn. Res..

[16]  Honglak Lee,et al.  Augmenting CRFs with Boltzmann Machine Shape Priors for Image Labeling , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Larry S. Davis,et al.  Detection, analysis and matching of hair , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[18]  Michael Harville,et al.  Automatic Skin Pixel Selection and Skin Color Classification , 2006, 2006 International Conference on Image Processing.

[19]  Zhenyun Peng,et al.  Image Based Hair Segmentation Algorithm for the Application of Automatic Facial Caricature Synthesis , 2014, TheScientificWorldJournal.