A Unified Framework for Head Pose, Age and Gender Classification through End-to-End Face Segmentation

Accurate face segmentation strongly benefits the human face image analysis problem. In this paper we propose a unified framework for face image analysis through end-to-end semantic face segmentation. The proposed framework contains a set of stack components for face understanding, which includes head pose estimation, age classification, and gender recognition. A manually labeled face data-set is used for training the Conditional Random Fields (CRFs) based segmentation model. A multi-class face segmentation framework developed through CRFs segments a facial image into six parts. The probabilistic classification strategy is used, and probability maps are generated for each class. The probability maps are used as features descriptors and a Random Decision Forest (RDF) classifier is modeled for each task (head pose, age, and gender). We assess the performance of the proposed framework on several data-sets and report better results as compared to the previously reported results.

[1]  Mahir Faik Karaaba,et al.  Deep Convolutional Neural Networks and Support Vector Machines for Gender Recognition , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[2]  Erik G. Learned-Miller,et al.  Towards unconstrained face recognition , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[3]  Massimo Mauro,et al.  Head pose estimation through multi-class face segmentation , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[4]  Alessandro Vinciarelli,et al.  Role Recognition in Broadcast News using Bernoulli Distributions , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[5]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[6]  Luc Van Gool,et al.  Real Time Head Pose Estimation from Consumer Depth Cameras , 2011, DAGM-Symposium.

[7]  Luc Van Gool,et al.  SEEDS: Superpixels Extracted Via Energy-Driven Sampling , 2012, International Journal of Computer Vision.

[8]  Stefan Fruehauf.  Perceiving And Remembering Faces , 2016 .

[9]  Yun Fu,et al.  Age Synthesis and Estimation via Faces: A Survey , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Michael Felsberg,et al.  Semantic Pyramids for Gender and Action Recognition , 2014, IEEE Transactions on Image Processing.

[11]  Huizhong Chen,et al.  The Hidden Sides of Names—Face Modeling with First Name Attributes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Alessandro Vinciarelli,et al.  Role recognition in multiparty recordings using social affiliation networks and discrete distributions , 2008, ICMI '08.

[13]  Shree K. Nayar,et al.  FaceTracer: A Search Engine for Large Collections of Images with Faces , 2008, ECCV.

[14]  Stefanos Zafeiriou,et al.  Robust Discriminative Response Map Fitting with Constrained Local Models , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Julia Hirschberg,et al.  The Rules Behind Roles: Identifying Speaker Role in Radio Broadcasts , 2000, AAAI/IAAI.

[16]  Karl Ricanek,et al.  MORPH: a longitudinal image database of normal adult age-progression , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[17]  Tal Hassner,et al.  Age and Gender Estimation of Unfiltered Faces , 2014, IEEE Transactions on Information Forensics and Security.

[18]  Jean-Marc Odobez,et al.  Predicting two facets of social verticality in meetings from five-minute time slices and nonverbal cues , 2008, ICMI '08.

[19]  Shiguang Shan,et al.  CovGa: A novel descriptor based on symmetry of regions for head pose estimation , 2014, Neurocomputing.

[20]  Rainer Stiefelhagen,et al.  DriveAHead — A Large-Scale Driver Head Pose Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Afshin Dehghan,et al.  DAGER: Deep Age, Gender and Emotion Recognition Using Convolutional Neural Network , 2017, ArXiv.

[22]  Massimo Mauro,et al.  Multi-class semantic segmentation of faces , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[23]  Canqun Yang,et al.  A hybrid deep learning CNN-ELM for age and gender classification , 2018, Neurocomputing.

[24]  Horst Bischof,et al.  Annotated Facial Landmarks in the Wild: A large-scale, real-world database for facial landmark localization , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[25]  Shumeet Baluja,et al.  Boosting Sex Identification Performance , 2005, International Journal of Computer Vision.

[26]  Tal Hassner,et al.  Effective face frontalization in unconstrained images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Alex Pentland,et al.  Modeling Dynamical Influence in Human Interaction: Using data to make better inferences about influence within social systems , 2012, IEEE Signal Processing Magazine.

[28]  Pritee Khanna,et al.  An illumination, expression, and noise invariant gender classifier using two-directional 2DPCA on real Gabor space , 2015, J. Comput. Lang..

[29]  Alessandro Vinciarelli.  Sociometry based Multiparty Audio Recordings Segmentation , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[30]  Yun Fu,et al.  Image-Based Human Age Estimation by Manifold Learning and Locally Adjusted Robust Regression , 2008, IEEE Transactions on Image Processing.

[31]  Tieniu Tan,et al.  A Study on Gait-Based Gender Classification , 2009, IEEE Transactions on Image Processing.

[32]  Trevor Darrell,et al.  PANDA: Pose Aligned Networks for Deep Attribute Modeling , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Xiaohui Yuan,et al.  Multi-level structured hybrid forest for joint head detection and pose estimation , 2017, Neurocomputing.

[34]  Mahmoud Afifi,et al.  AFIF4: Deep Gender Classification based on AdaBoost-based Fusion of Isolated Facial Features and Foggy Faces , 2017, J. Vis. Commun. Image Represent..

[35]  Rama Chellappa,et al.  Modeling Age Progression in Young Faces , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[36]  James L. Crowley,et al.  Head Pose Estimation Using Multi-scale Gaussian Derivatives , 2013, SCIA.

[37]  Luc Van Gool,et al.  Real-time facial feature detection using conditional regression forests , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  Ming-Hsuan Yang,et al.  Learning Gender with Support Faces , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Matthew Toews,et al.  Detection, Localization, and Sex Classification of Faces from Arbitrary Viewpoints and under Occlusion , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Stefanos Zafeiriou,et al.  Incremental Face Alignment in the Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Qiang Ji,et al.  Simultaneous Facial Landmark Detection, Pose and Deformation Estimation Under Facial Occlusion , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[42]  Larry S. Davis,et al.  On partial least squares in head pose estimation: How to simultaneously deal with misalignment , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Julia Hirschberg,et al.  Automatic summarization of broadcast news using structural features , 2003, INTERSPEECH.

[44]  Niels da Vitoria Lobo,et al.  Age Classification from Facial Images , 1999, Comput. Vis. Image Underst..

[45]  Marco La Cascia,et al.  Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Harry Wechsler,et al.  The FERET database and evaluation procedure for face-recognition algorithms , 1998, Image Vis. Comput..

[47]  Jian Sun,et al.  Face Alignment by Explicit Shape Regression , 2012, International Journal of Computer Vision.

[48]  Dilek Z. Hakkani-Tür,et al.  Role recognition for meeting participants: an approach based on lexical information and social network analysis , 2008, ACM Multimedia.

[49]  Tal Hassner,et al.  Age and gender classification using convolutional neural networks , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[50]  Pawan Sinha,et al.  Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About , 2006, Proceedings of the IEEE.

[51]  Terrence J. Sejnowski,et al.  SEXNET: A Neural Network Identifies Sex From Human Faces , 1990, NIPS.

[52]  Khalil Khan,et al.  Automatic Gender Classification through Face Segmentation , 2019, Symmetry.

[53]  Alessandro Vinciarelli,et al.  Automatic role recognition based on conversational and prosodic behaviour , 2010, ACM Multimedia.

[54]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[55]  Xin Geng,et al.  Head Pose Estimation Based on Multivariate Label Distribution , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Claudio A. Perez,et al.  Gender Classification Based on Fusion of Different Spatial Scale Features Selected by Mutual Information From Histogram of LBP, Intensity, and Shape , 2013, IEEE Transactions on Information Forensics and Security.

[57]  Changsheng Li,et al.  Human Age Estimation Based on Locality and Ordinal Information , 2015, IEEE Transactions on Cybernetics.

[58]  Simon Lucey,et al.  Deformable Model Fitting by Regularized Landmark Mean-Shift , 2010, International Journal of Computer Vision.

[59]  Sheng Wan,et al.  QuatNet: Quaternion-Based Head Pose Estimation With Multiregression Loss , 2019, IEEE Transactions on Multimedia.

[60]  Saeed Mozaffari,et al.  Gender dictionary learning for gender classification , 2017, J. Vis. Commun. Image Represent..

[61]  Roope Raisamo,et al.  Evaluation of Gender Classification Methods with Automatically Detected and Aligned Faces , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Carlos E. Thomaz,et al.  A Priori-Driven PCA , 2012, ACCV Workshops.

[63]  Р Ю Чуйков,et al.  Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector , 2017 .

[64]  Peter Robinson,et al.  3D Constrained Local Model for rigid and non-rigid facial tracking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Dimitris Samaras,et al.  Squared Earth Mover's Distance-based Loss for Training Deep Neural Networks , 2016, ArXiv.

[66]  Alexander Binder,et al.  Understanding and Comparing Deep Neural Networks for Age and Gender Classification , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[67]  James M. Rehg,et al.  Fine-Grained Head Pose Estimation Without Keypoints , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[68]  Chu-Song Chen,et al.  A Learning Framework for Age Rank Estimation Based on Face Images With Scattering Transform , 2015, IEEE Transactions on Image Processing.

[69]  Ralph Gross,et al.  Generic vs. person specific active appearance models , 2005, Image Vis. Comput..

[70]  J. Crowley,et al.  Estimating Face orientation from Robust Detection of Salient Facial Structures , 2004 .

[71]  Peter Robinson,et al.  OpenFace: An open source facial behavior analysis toolkit , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[72]  Louis-Philippe Morency,et al.  OpenFace 2.0: Facial Behavior Analysis Toolkit , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[73]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[74]  Alexander I. Rudnicky,et al.  Using simple speech-based features to detect the state of a meeting and the roles of the meeting participants , 2004, INTERSPEECH.

[75]  Alex Pentland,et al.  Modeling Functional Roles Dynamics in Small Group Interactions , 2013, IEEE Transactions on Multimedia.

[76]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[77]  Takeshi Saitoh,et al.  Head Pose Estimation Using Convolutional Neural Network , 2018 .

[78]  Wei-Ta Chu,et al.  Movie Analysis Based on Roles' Social Network , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[79]  Maizatul Akmar Ismail,et al.  Face Recognition and Age Estimation Implications of Changes in Facial Features: A Critical Review Study , 2018, IEEE Access.

[80]  David J. Kriegman,et al.  Localizing Parts of Faces Using a Consensus of Exemplars , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  Xiaoming Liu,et al.  Demographic Estimation from Face Images: Human vs. Machine Performance , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[82]  Feiping Nie,et al.  Compound Rank- $k$ Projections for Bilinear Analysis , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[83]  Massimo Mauro,et al.  Gender and Expression Analysis Based on Semantic Face Segmentation , 2017, ICIAP.

[84]  Rama Chellappa,et al.  HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.