Gender Estimation Based on Smile-Dynamics

Automated gender estimation has numerous applications, including video surveillance, human–computer interaction, anonymous customized advertisement, and image retrieval. Most commonly, the underlying algorithms analyze the facial appearance for clues of gender. In this paper, we propose a novel method for gender estimation, which exploits dynamic features gleaned from smiles and we proceed to show that: a) facial dynamics incorporate clues for gender dimorphism and b) while for adult individuals appearance features are more accurate than dynamic features, for subjects under 18 years facial dynamics can outperform appearance features. In addition, we fuse proposed dynamics-based approach with state-of-the-art appearance-based algorithms, predominantly improving performance of the latter. Results show that smile-dynamics include pertinent and complementary to appearance gender information.

[1]  Stefanos Zafeiriou,et al.  300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[2]  Bin Xia,et al.  Multi-view gender classification based on local Gabor binary mapping pattern and support vector machines , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[3]  Jean-Luc Dugelay,et al.  Image-based gender estimation from body and face across distances , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[4]  Leda E. Nath,et al.  Gender and Emotion in the United States: Do Men and Women Differ in Self‐Reports of Feelings and Expressive Behavior?1 , 2004, American Journal of Sociology.

[5]  Matthew Toews,et al.  Gender classification from unconstrained video sequences , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[6]  Aleix M. Martínez,et al.  A Model of the Perception of Facial Expressions of Emotion by Humans: Research Overview and Perspectives , 2012, J. Mach. Learn. Res..

[7]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[8]  Marios Savvides,et al.  DeepGender: Occlusion and Low Resolution Robust Facial Gender Classification via Progressively Trained Convolutional Neural Networks with Attention , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[10]  Jean-Luc Dugelay,et al.  Search pruning in video surveillance systems: Efficiency-reliability tradeoff , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[11]  Hassan Ugail,et al.  Gender Classification Based on 3D Face Geometry Features Using SVM , 2009, 2009 International Conference on CyberWorlds.

[12]  Anil K. Jain,et al.  Face Recognition Performance: Role of Demographic Information , 2012, IEEE Transactions on Information Forensics and Security.

[13]  Marios Savvides,et al.  Fusion of region-based representations for gender identification , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[14]  Nello Cristianini,et al.  Learning to classify gender from four million images , 2015, Pattern Recognit. Lett..

[15]  José Miguel Buenaposada,et al.  Revisiting Linear Discriminant Techniques in Gender Recognition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Arun Ross,et al.  Impact of facial cosmetics on automatic gender and age estimation algorithms , 2014, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[17]  G. Mather,et al.  Gender discrimination in biological motion displays based on dynamic cues , 1994, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[18]  Shumeet Baluja,et al.  Boosting Sex Identification Performance , 2005, International Journal of Computer Vision.

[19]  Arun Ross,et al.  Can Gender Be Predicted from Near-Infrared Face Images? , 2011, ICIAR.

[20]  Matti Pietikäinen,et al.  On the use of dynamic features in face biometrics: recent advances and challenges , 2011, Signal Image Video Process..

[21]  Arun Ross,et al.  Can facial metrology predict gender? , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[22]  Jean-Luc Dugelay,et al.  Bag of soft biometrics for person identification , 2010, Multimedia Tools and Applications.

[23]  C. Malatesta,et al.  The development of emotion expression during the first two years of life. , 1989, Monographs of the Society for Research in Child Development.

[24]  Antitza Dantcheva,et al.  Can a Smile Reveal Your Gender? , 2016, 2016 International Conference of the Biometrics Special Interest Group (BIOSIG).

[25]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Jean-Luc Dugelay,et al.  Person recognition using a bag of facial soft biometrics (BoFSB) , 2010, 2010 IEEE International Workshop on Multimedia Signal Processing.

[27]  M. Arfan Jaffar,et al.  Feature selection for efficient gender classification , 2010 .

[28]  Stefanos Zafeiriou,et al.  Incremental Face Alignment in the Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  A. Aldao,et al.  Gender differences in emotion expression in children: a meta-analytic review. , 2013, Psychological bulletin.

[30]  Matti Pietikäinen,et al.  Combining appearance and motion for face and gender recognition from videos , 2009, Pattern Recognit..

[31]  Sebastian Korb,et al.  The two sides of spontaneity: Movement onset asymmetries in facial expressions influence social judgments , 2014 .

[32]  Anil K. Jain,et al.  Can soft biometric traits assist user recognition? , 2004, SPIE Defense + Commercial Sensing.

[33]  Caifeng Shan,et al.  Gender Classification on Real-Life Faces , 2010, ACIVS.

[34]  K. Heilman,et al.  Digitizing the moving face during dynamic displays of emotion , 2000, Neuropsychologia.

[35]  Dominique Valentin,et al.  Sex classification of face areas: How well can a linear neural network predict human performance? , 1998 .

[36]  Albert Ali Salah,et al.  Are You Really Smiling at Me? Spontaneous versus Posed Enjoyment Smiles , 2012, ECCV.

[37]  Masatoshi Kawai,et al.  Mother-infant face-to-face interaction in Japan and the United States: A laboratory comparison using 3-month-old infants. , 1988 .

[38]  P. Ekman,et al.  Felt, false, and miserable smiles , 1982 .

[39]  Arun Ross,et al.  Evaluation of gender classification methods on thermal and near-infrared face images , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[40]  Judith A. Hall,et al.  Gender and emotion in context. , 2008 .

[41]  Caifeng Shan,et al.  Learning local binary patterns for gender classification on real-world face images , 2012, Pattern Recognit. Lett..

[42]  Lijun Yin,et al.  FERA 2015 - second Facial Expression Recognition and Analysis challenge , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[43]  F. Deutsch,et al.  What is in a Smile? , 1987 .

[44]  Albert Ali Salah,et al.  A smile can reveal your age: enabling facial dynamics in age estimation , 2012, ACM Multimedia.

[45]  P. Ekman.  Facial expression and emotion. , 1993, The American psychologist.

[46]  Javier Lorenzo-Navarro,et al.  Gender Classification in Large Databases , 2012, CIARP.

[47]  Reginald B. Adams,et al.  The Intersection of Gender-Related Facial Appearance and Facial Displays of Emotion , 2015 .

[48]  Paul Scheunders,et al.  Advanced Concepts for Intelligent Vision Systems , 2018, Lecture Notes in Computer Science.

[49]  José Miguel Buenaposada,et al.  Robust gender recognition by exploiting facial attributes dependencies , 2014, Pattern Recognit. Lett..

[50]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Albert Ali Salah,et al.  Recognition of Genuine Smiles , 2015, IEEE Transactions on Multimedia.

[52]  Anil K. Jain,et al.  Open source biometric recognition , 2013, 2013 IEEE Sixth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[53]  Lijun Yin,et al.  Static and dynamic 3D facial expression recognition: A comprehensive survey , 2012, Image Vis. Comput..

[54]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[55]  A. Dantcheva,et al.  Bag of Soft Biometrics for Person Identification New trends and challenges , 2011 .

[56]  Shiqing Zhang,et al.  Facial Expression Recognition via Deep Learning , 2015 .

[57]  Albert Ali Salah,et al.  Combining Facial Dynamics With Appearance for Age Estimation , 2015, IEEE Transactions on Image Processing.

[58]  Roope Raisamo,et al.  Evaluation of Gender Classification Methods with Automatically Detected and Aligned Faces , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Reginald B. Adams,et al.  Who may frown and who should smile? Dominance, affiliation, and the display of happiness and anger , 2005 .

[60]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Marie-Françoise Liébart,et al.  Smile Line and Periodontium Visibility , 2004 .

[62]  Leslie A. Zebrowitz,et al.  Social Psychological Face Perception: Why Appearance Matters. , 2008, Social and personality psychology compass.

[63]  J. Haviland,et al.  Learning display rules: the socialization of emotion expression in infancy. , 1982, Child development.

[64]  Arun Ross,et al.  Soft biometrics for surveillance: an overview , 2013 .

[65]  Reginald B. Adams,et al.  Facial appearance, gender, and emotion expression. , 2004, Emotion.

[66]  Yun Fu,et al.  Is gender recognition affected by age? , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[67]  E. Cashdan.  Smiles, Speech, and Body Posture: How Women and Men Display Sociometric Status and Power , 1998 .

[68]  Arun Ross,et al.  What Else Does Your Biometric Data Reveal? A Survey on Soft Biometrics , 2016, IEEE Transactions on Information Forensics and Security.

[69]  R. Chellappa,et al.  Age progression in Human Faces : A Survey , 2008 .

[70]  O. Pascalis,et al.  Representation of the Gender of Human Faces by Infants: A Preference for Female , 2002, Perception.

[71]  Steven L. Neuberg,et al.  The confounded nature of angry men and happy women. , 2007, Journal of personality and social psychology.

[72]  Wei Gao,et al.  Face Gender Classification on Consumer Images in a Multiethnic Environment , 2009, ICB.

[73]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[74]  Li Zhuo,et al.  Learning realistic facial expressions from web images , 2013, Pattern Recognit..

[75]  Ursula Hess,et al.  Why the Same Expression May Not Mean the Same When Shown on Different Faces or Seen by Different People , 2009, Affective Information Processing.

[76]  Jean-Luc Dugelay,et al.  Soft biometrics systems: Reliability and asymptotic bounds , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[77]  Bok-Min Goi,et al.  Vision-based Human Gender Recognition: A Survey , 2012, ArXiv.

[78]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  Paul F. Velleman,et al.  Definition and Comparison of Robust Nonlinear Data Smoothing Algorithms , 1980 .

[80]  James W. Tanaka,et al.  Angry facial expressions bias gender categorization in children and adults: behavioral and computational evidence , 2015, Front. Psychol..

[81]  Jun Li,et al.  Boosting dense SIFT descriptors and shape contexts of face images for gender recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[82]  A. O'Toole,et al.  An ‘other-Race Effect’ for Categorizing Faces by Sex , 1996, Perception.

[83]  Matthew Toews,et al.  Detection, Localization, and Sex Classification of Faces from Arbitrary Viewpoints and under Occlusion , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[84]  Reginald B. Adams,et al.  When Two Do the Same, It Might Not Mean the Same: The Perception of Emotional Expressions Shown by Men and Women. , 2007 .

[85]  J. Langlois,et al.  Attractive Faces Are Only Average , 1990 .