Physiognomy: Personality traits prediction by learning

Evaluating individuals’ personality traits and intelligence from their faces plays a crucial role in interpersonal relationship and important social events such as elections and court sentences. To assess the possible correlations between personality traits (also measured intelligence) and face images, we first construct a dataset consisting of face photographs, personality measurements, and intelligence measurements. Then, we build an end-to-end convolutional neural network for prediction of personality traits and intelligence to investigate whether self-reported personality traits and intelligence can be predicted reliably from a face image. To our knowledge, it is the first work where deep learning is applied to this problem. Experimental results show the following three points: 1) “Rule-consciousness” and “Tension” can be reliably predicted from face images. 2) It is difficult, if not impossible, to predict intelligence from face images, a finding in accord with previous studies. 3) Convolutional neural network (CNN) features outperform traditional handcrafted features in predicting traits.

[1]  K. Barton.  A guide to the clinical use of the 16 PF. , 1978 .

[2]  D. Saccuzzo,et al.  Psychological Testing: Principles, Applications, and Issues , 1982 .

[3]  D. McNeill.  The Face: A Natural History , 1998 .

[4]  Chengjun Liu,et al.  Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition , 2002, IEEE Trans. Image Process..

[5]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[6]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[7]  C. Judd,et al.  The Influence of Afrocentric Facial Features in Criminal Sentencing , 2004, Psychological science.

[8]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[9]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  A. Todorov,et al.  Predicting political elections from rapid and unreflective face judgments , 2007, Proceedings of the National Academy of Sciences.

[11]  Susan C. Roberts,et al.  Facial appearance affects voting decisions. , 2007 .

[12]  Judith A. Hall,et al.  A thin slice perspective on the accuracy of first impressions , 2007 .

[13]  A. Todorov,et al.  The functional basis of face evaluation , 2008, Proceedings of the National Academy of Sciences.

[14]  Andrew D. Engell,et al.  Understanding evaluation of faces on social dimensions , 2008, Trends in Cognitive Sciences.

[15]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[16]  Tal Hassner,et al.  Multiple One-Shots for Utilizing Class Label Information , 2009, BMVC.

[17]  Cordelia Schmid,et al.  Is that you? Metric learning approaches for face identification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[18]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[19]  David Masip,et al.  Automatic point-based facial trait judgments evaluation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Jordi Vitrià,et al.  Automatic Prediction of Facial Trait Judgments: Appearance vs. Structural Models , 2011, PloS one.

[21]  J. King,et al.  Identifying personality from the static, nonexpressive face in humans and chimpanzees: evidence of a shared system for signaling personality , 2011 .

[22]  Jian Sun,et al.  An associate-predict model for face recognition , 2011, CVPR 2011.

[23]  Shenghuo Zhu,et al.  Large Scale Strongly Supervised Ensemble Metric Learning, with Applications to Face Verification and Retrieval , 2012, ArXiv.

[24]  Peter N. Belhumeur,et al.  Tom-vs-Pete Classifiers and Identity-Preserving Alignment for Face Verification , 2012, BMVC.

[25]  Jian Sun,et al.  Bayesian Face Revisited: A Joint Formulation , 2012, ECCV.

[26]  Jian Sun,et al.  Blessing of Dimensionality: High-Dimensional Feature and Its Efficient Compression for Face Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Xiaogang Wang,et al.  Deep Learning Identity-Preserving Face Space , 2013, 2013 IEEE International Conference on Computer Vision.

[28]  Jian Sun,et al.  A Practical Transfer Learning Algorithm for Face Verification , 2013, 2013 IEEE International Conference on Computer Vision.

[29]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[30]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  J. Flegr,et al.  Perceived Intelligence Is Associated with Measured Intelligence in Men but Not Women , 2014, PloS one.

[32]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[33]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[34]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Ulrik Plesner Jacobsen,et al.  Interpretation of Appearance: The Effect of Facial Features on First Impressions and Personality , 2014, PloS one.

[36]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[37]  Michael I. Jordan,et al.  Learning Transferable Features with Deep Adaptation Networks , 2015, ICML.

[38]  Ming Yang,et al.  Web-scale training for face identification , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Ting Zhang,et al.  Shape initialization without ground truth for face alignment , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Amir Akramin Shafie,et al.  Robust face recognition against expressions and partial occlusions , 2016, Int. J. Autom. Comput..

[41]  Zhanyi Hu,et al.  Pursuing face identity from view-specific representation to view-invariant representation , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[42]  Guanrong Chen,et al.  Pinning control and controllability of complex dynamical networks , 2016, International Journal of Automation and Computing.

[43]  Shuicheng Yan,et al.  A survey on deep learning-based fine-grained object classification and semantic segmentation , 2017, International Journal of Automation and Computing.