Demographic Estimation from Face Images: Human vs. Machine Performance

Demographic estimation entails automatic estimation of age, gender and race of a person from his face image, which has many potential applications ranging from forensics to social media. Automatic demographic estimation, particularly age estimation, remains a challenging problem because persons belonging to the same demographic group can be vastly different in their facial appearances due to intrinsic and extrinsic factors. In this paper, we present a generic framework for automatic demographic (age, gender and race) estimation. Given a face image, we first extract demographic informative features via a boosting algorithm, and then employ a hierarchical approach consisting of between-group classification, and within-group regression. Quality assessment is also developed to identify low-quality face images that are difficult to obtain reliable demographic estimates. Experimental results on a diverse set of face image databases, FG-NET (1K images), FERET (3K images), MORPH II (75K images), PCSO (100K images), and a subset of LFW (4K images), show that the proposed approach has superior performance compared to the state of the art. Finally, we use crowdsourcing to study the human perception ability of estimating demographics from face images. A side-by-side comparison of the demographic estimates from crowdsourced data and the proposed algorithm provides a number of insights into this challenging problem.

[1]  D. Perrett,et al.  What Gives a Face its Gender? , 1993, Perception.

[2]  Yun Fu,et al.  Human Age Estimation With Regression on Discriminative Aging Manifold , 2008, IEEE Transactions on Multimedia.

[3]  Edwin R. Hancock,et al.  Facial gender classification using shape-from-shading , 2010, Image Vis. Comput..

[4]  Anil K. Jain,et al.  Unconstrained face recognition: Establishing baseline human performance via crowdsourcing , 2014, IEEE International Joint Conference on Biometrics.

[5]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[6]  Wen Gao,et al.  Design sparse features for age estimation using hierarchical face model , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[7]  Yun Fu,et al.  Human age estimation using bio-inspired features , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Yunus Saatci,et al.  Cascaded classification of gender and facial expression using active appearance models , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[9]  Brunelli Poggio,et al.  HyberBF Networks for Gender Classification , 1992 .

[10]  Timothy F. Cootes,et al.  Toward Automatic Simulation of Aging Effects on Face Images , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  José Miguel Buenaposada,et al.  Revisiting Linear Discriminant Techniques in Gender Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Shlomo Bentin,et al.  The role of features and configural processing in face-race classification , 2011, Vision Research.

[13]  Thomas Serre,et al.  Object recognition with features inspired by visual cortex , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  J. Hayashi,et al.  Age and gender estimation based on wrinkle texture and color of facial images , 2002, Object recognition supported by user interaction for service robots.

[15]  Margo Seltzer,et al.  The Mug-Shot Search Problem , 1997 .

[16]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[17]  Zhi-Hua Zhou,et al.  Automatic Age Estimation Based on Facial Aging Patterns , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Arun Ross,et al.  Can facial metrology predict gender? , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[19]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[20]  Yunhong Wang,et al.  Hierarchical and discriminative bag of features for face profile and ear based gender classification , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[21]  Anil K. Jain,et al.  Handbook of Face Recognition, 2nd Edition , 2011 .

[22]  Haizhou Ai,et al.  Demographic Classification with Local Binary Patterns , 2007, ICB.

[23]  Yi-Ping Hung,et al.  Ordinal hyperplanes ranker with cost sensitivities for age estimation , 2011, CVPR 2011.

[24]  D. Perrett,et al.  Perception of age in adult Caucasian male faces: computer graphic manipulation of shape and colour information , 1995, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[25]  Alice J. O'Toole,et al.  Comparison of human and computer performance across face recognition experiments , 2014, Image and Vision Computing.

[26]  Yuxiao Hu,et al.  Gender and ethnicity identification from silhouetted face profiles , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[27]  Saeed Mozaffari,et al.  Gender Classification Using Single Frontal Image Per Person: Combination of Appearance and Geometric Based Features , 2010, 2010 20th International Conference on Pattern Recognition.

[28]  H. Ai,et al.  LUT-Based Adaboost for Gender Classification , 2003, AVBPA.

[29]  Shree K. Nayar,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Describable Visual Attributes for Face Verification and Image Search , 2022 .

[30]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Driss Aboutajdine,et al.  Boosting 3-D-Geometric Features for Efficient Face Recognition and Gender Classification , 2012, IEEE Transactions on Information Forensics and Security.

[32]  Ming-Hsuan Yang,et al.  Gender classification with support vector machines , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[33]  Anil K. Jain,et al.  Age estimation from face images: Human vs. machine performance , 2013, 2013 International Conference on Biometrics (ICB).

[34]  Anil K. Jain,et al.  Multimodal Facial Gender and Ethnicity Identification , 2006, ICB.

[35]  Jiwen Lu,et al.  Ordinary Preserving Manifold Analysis for Human Age and Head Pose Estimation , 2013, IEEE Transactions on Human-Machine Systems.

[36]  Garrison W. Cottrell,et al.  EMPATH: Face, Emotion, and Gender Recognition Using Holons , 1990, NIPS.

[37]  Rama Chellappa,et al.  A hierarchical approach for human age estimation , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[38]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[39]  Niels da Vitoria Lobo,et al.  Age Classification from Facial Images , 1999, Comput. Vis. Image Underst..

[40]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[41]  David G. Lowe,et al.  University of British Columbia. , 1945, Canadian Medical Association journal.

[42]  Tim Oates,et al.  Joint inference of soft biometric features , 2012, 2012 5th IAPR International Conference on Biometrics (ICB).

[43]  Denise C. Park,et al.  A lifespan database of adult facial stimuli , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[44]  Arun Ross,et al.  Local gradient Gabor pattern (LGGP) with applications in face recognition, cross-spectral matching, and soft biometrics , 2013, Defense, Security, and Sensing.

[45]  Terrance E. Boult,et al.  Multi-attribute spaces: Calibration for attribute fusion and similarity search , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Wei Gao,et al.  Face Gender Classification on Consumer Images in a Multiethnic Environment , 2009, ICB.

[47]  Shumeet Baluja,et al.  Boosting Sex Identification Performance , 2005, International Journal of Computer Vision.

[48]  Elham Tabassi,et al.  Performance of Biometric Quality Measures , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Zhi-Hua Zhou,et al.  Facial Age Estimation by Learning from Label Distributions , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Matthew G. Rhodes,et al.  Age estimation of faces: a review† , 2009 .

[51]  Elham Tabassi,et al.  Fingerprint Image Quality , 2009, Encyclopedia of Biometrics.

[52]  Yun Fu,et al.  Age Synthesis and Estimation via Faces: A Survey , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Kang Ryoung Park,et al.  Age estimation using a hierarchical classifier based on global and local facial features , 2011, Pattern Recognit..

[54]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[55]  Lior Wolf,et al.  Using Biologically Inspired Features for Face Processing , 2007, International Journal of Computer Vision.

[56]  Guodong Guo,et al.  Joint estimation of age, gender and ethnicity: CCA vs. PLS , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[57]  Timothy F. Cootes,et al.  Active Appearance Models , 1998, ECCV.

[58]  Rama Chellappa,et al.  Age Estimation and Face Verification Across Aging Using Landmarks , 2012, IEEE Transactions on Information Forensics and Security.

[59]  Jian-Jiun Ding,et al.  Facial age estimation based on label-sensitive learning and age-oriented regression , 2013, Pattern Recognit..

[60]  Cuixian Chen,et al.  Gender classification from infants to seniors , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[61]  Karl Ricanek,et al.  MORPH: a longitudinal image database of normal adult age-progression , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[62]  L. Semaj,et al.  The Development of Racial-Classification Abilities. , 1981 .

[63]  Arun Ross,et al.  Evaluation of gender classification methods on thermal and near-infrared face images , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[64]  Takashi Kitahara,et al.  Comparison of age-related changes in facial wrinkles and sagging in the skin of Japanese, Chinese and Thai women. , 2007, Journal of dermatological science.

[65]  Claudio A. Perez,et al.  Gender Classification Based on Fusion of Different Spatial Scale Features Selected by Mutual Information From Histogram of LBP, Intensity, and Shape , 2013, IEEE Transactions on Information Forensics and Security.

[66]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[67]  Roope Raisamo,et al.  An experimental comparison of gender classification methods , 2008, Pattern Recognit. Lett..

[68]  Harry Wechsler,et al.  Mixture of experts for classification of gender, ethnic origin, and pose of human faces , 2000, IEEE Trans. Neural Networks Learn. Syst..

[69]  Yujie Dong,et al.  Eyebrow shape-based features for biometric recognition and gender classification: A feasibility study , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[70]  Yajie Tian,et al.  Handbook of face recognition , 2003 .

[71]  Kenneth O. May,et al.  A Set of Independent Necessary and Sufficient Conditions for Simple Majority Decision , 1952 .

[72]  Natalie C. Ebner,et al.  Let me guess how old you are: effects of age, gender, and facial expression on perceptions of age. , 2012, Psychology and aging.

[73]  Guodong Guo,et al.  Simultaneous dimensionality reduction and human age estimation via kernel partial least squares regression , 2011, CVPR 2011.

[74]  Matti Pietikäinen,et al.  Demographic classification from face videos using manifold learning , 2013, Neurocomputing.

[75]  Chengjun Liu,et al.  Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition , 2002, IEEE Trans. Image Process..

[76]  Ching Y. Suen,et al.  Contourlet appearance model for facial age estimation , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[77]  Rama Chellappa,et al.  Modeling Age Progression in Young Faces , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).