Social profiling through image understanding: Personality inference using convolutional neural networks

Linking OCEAN personality traits and preferred images in the Flickr social network.Classification of personality traits by novel image features, designed by CNN.Interpretation of visual features by an ad-hoc deconvolution strategy.Online demo application. The role of images in the last ten years has changed radically due to the advent of social networks: from media objects mainly used to communicate visual information, images have become personal, associated with the people that create or interact with them (for example, giving a like). Therefore, in the same way that a post reveals something of its author, so now the images associated to a person may embed some of her individual characteristics, such as her personality traits. In this paper, we explore this new level of image understanding with the ultimate goal of relating a set of image preferences to personality traits by using a deep learning framework. In particular, our problem focuses on inferring both self-assessed (how the personality traits of a person can be guessed from her preferred image) and attributed traits (what impressions in terms of personality traits these images trigger in unacquainted people), learning a sort of wisdom of the crowds. Our characterization of each image is locked within the layers of a CNN, allowing us to discover more entangled attributes (aesthetic patterns and semantic information) and to better generalize the patterns that identify a trait. The experimental results show that the proposed method outperforms state-of-the-art results and captures what visually characterizes a certain trait: using a deconvolution strategy we found a clear distinction of features, patterns and content between low and high values in a given trait.