Integrating Vision and Language for First-Impression Personality Analysis

The authors present a novel methodology for analyzing integrated audiovisual signals and language to assess a persons personality. An evaluation of their proposed multimodal method using a job candidate screening system that predicted five personality traits from a short video demonstrates the methods effectiveness.

[1]  Albert Ali Salah,et al.  Multi-modal Score Fusion and Decision Trees for Explainable Automatic Job Candidate Screening from Video CVs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Sergio Escalera,et al.  ChaLearn LAP 2016: First Round Challenge on First Impressions - Dataset and Results , 2016, ECCV Workshops.

[3]  Margaret Lech,et al.  Evaluating deep learning architectures for Speech Emotion Recognition , 2017, Neural Networks.

[4]  Gholamreza Anbarjafari,et al.  Automated Screening of Job Candidate Based on Multimodal Video Processing , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[5]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[6]  Subramanian Ramanathan,et al.  Automatic modeling of personality states in small group interactions , 2011, MM '11.

[7]  S. Srivastava,et al.  The Big Five Trait taxonomy: History, measurement, and theoretical perspectives. , 1999 .

[8]  Tim Polzehl,et al.  Automatically Assessing Personality from Speech , 2010, 2010 IEEE Fourth International Conference on Semantic Computing.

[9]  Marie-Francine Moens,et al.  Vision and Language Integration Meets Multimedia Fusion , 2018, IEEE Multim..

[10]  Sergio Escalera,et al.  Audio-Visual Emotion Recognition in Video Clips , 2019, IEEE Transactions on Affective Computing.

[11]  Sergio Escalera,et al.  Multimodal First Impression Analysis with Deep Residual Networks , 2018, IEEE Transactions on Affective Computing.

[12]  Mohd Heikal Husin,et al.  Sentiment Valences for Automatic Personality Detection of Online Social Networks Users Using Three Factor Model , 2015 .

[13]  A. Rogier.  [Communication without words]. , 1971, Tijdschrift voor ziekenverpleging.

[14]  Alessandro Vinciarelli,et al.  The voice of personality: mapping nonverbal vocal behavior into trait attributions , 2010, SSPW '10.

[15]  Marco Cristani,et al.  Social profiling through image understanding: Personality inference using convolutional neural networks , 2017, Comput. Vis. Image Underst..

[16]  Daniel Gatica-Perez,et al.  The YouTube Lens: Crowdsourced Personality Impressions and Audiovisual Analysis of Vlogs , 2013, IEEE Transactions on Multimedia.

[17]  Trevor Darrell,et al.  Long-term recurrent convolutional networks for visual recognition and description , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).