Like Partying? Your Face Says It All. Predicting the Ambiance of Places with Profile Pictures

To choose restaurants and coffee shops, people are increasingly relying on social-networking sites. In a popular site such as Foursquare or Yelp, a place comes with descriptions and reviews, and with profile pictures of people who frequent them. Descriptions and reviews have been widely explored in the research area of data mining. By contrast, profile pictures have received little attention. Previous work showed that people are able to partly guess a place's ambiance, clientele, and activities not only by observing the place itself but also by observing the profile pictures of its visitors. Here we further that work by determining which visual cues people may have relied upon to make their guesses; showing that a state-of-the-art algorithm could make predictions more accurately than humans at times; and demonstrating that the visual cues people relied upon partly differ from those of the algorithm.

[1]  Christopher Y. Olivola,et al.  Elected in 100 milliseconds: Appearance-Based Trait Inferences and Voting , 2010 .

[2]  Lois B. Wexner The degree to which colors (hues) are associated with mood-tones. , 1954 .

[3]  J. G. Snodgrass,et al.  A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. , 1980, Journal of experimental psychology. Human learning and memory.

[4]  J. Concato,et al.  A simulation study of the number of events per variable in logistic regression analysis. , 1996, Journal of clinical epidemiology.

[5]  Yuning Jiang,et al.  Extensive Facial Landmark Localization with Coarse-to-Fine Convolutional Network Cascade , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[6]  Samuel D. Gosling,et al.  Can the Ambiance of a Place be Determined by the User Profiles of the People Who Visit It? , 2021, ICWSM.

[7]  Frank H. Mahnke,et al.  Color, Environment, and Human Response: An Interdisciplinary Understanding of Color and Its Use As a Beneficial Element in the Design of the Architectural Environment , 1996 .

[8]  Allan Hanbury,et al.  Affective image classification using features inspired by psychology and art theory , 2010, ACM Multimedia.

[9]  Rossano Schifanella,et al.  6 Seconds of Sound and Vision: Creativity in Micro-videos , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Soraya Mehdizadeh,et al.  Self-Presentation 2.0: Narcissism and Self-Esteem on Facebook , 2010, Cyberpsychology Behav. Soc. Netw..

[11]  Bill Bishop,et al.  The Big Sort: Why the Clustering of Like-Minded America Is Tearing Us Apart , 2008 .

[12]  Virgílio A. F. Almeida,et al.  Tips, dones and todos: uncovering user profiles in foursquare , 2012, WSDM '12.

[13]  Eric Gilbert,et al.  Faces engage us: photos with faces attract more likes and comments on Instagram , 2014, CHI.

[14]  Mohammed Yeasin,et al.  FEPS: a sensory substitution system for the blind to perceive facial expressions , 2012, ASSETS '12.

[15]  A. Goldman,et al.  Simulationist models of face-based emotion recognition , 2005, Cognition.

[16]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[17]  Jussara M. Almeida,et al.  Popularity dynamics of foursquare micro-reviews , 2014, COSN '14.

[18]  Bernard Mérialdo,et al.  Where is the beauty?: retrieving appealing VideoScenes by learning Flickr-based graded judgments , 2012, ACM Multimedia.

[19]  S. Gosling,et al.  Personality and Social Psychology Bulletin Personality Judgments Based on Physical Appearance Personality Judgments Based on Physical Appearance , 2022 .

[20]  James Ze Wang,et al.  On shape and the computability of emotions , 2012, ACM Multimedia.

[21]  James Ze Wang,et al.  Studying Aesthetics in Photographic Images Using a Computational Approach , 2006, ECCV.

[22]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[23]  Joshua B. Tenenbaum,et al.  Learning with Hierarchical-Deep Models , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  W. T. James,et al.  The Effect of Color Shock on Motor Performance and Tremor , 1953 .

[25]  Miriam Redi,et al.  The beauty of capturing faces: Rating the quality of digital portraits , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[26]  J C Mazziotta,et al.  Reafferent copies of imitated actions in the right superior temporal cortex , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[27]  N. Raz,et al.  Life Span Adult Faces: Norms for Age, Familiarity, Memorability, Mood, and Picture Quality , 2009, Experimental aging research.

[28]  M. Hemphill,et al.  A note on adults' color-emotion associations. , 1996, The Journal of genetic psychology.

[29]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[30]  Vagia Tsiminaki,et al.  Hi YouTube!: personality impressions and verbal content in social video , 2013, ICMI '13.

[31]  Luc Van Gool,et al.  The Interestingness of Images , 2013, 2013 IEEE International Conference on Computer Vision.

[32]  Daniel Gatica-Perez,et al.  FaceTube: predicting personality from facial expressions of emotion in online conversational video , 2012, ICMI '12.

[33]  Henriette Cramer,et al.  Aesthetic capital: what makes london look beautiful, quiet, and happy? , 2014, CSCW.

[34]  Song-Chun Zhu,et al.  Visual Persuasion: Inferring Communicative Intents of Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  P. Valdez,et al.  Effects of color on emotions. , 1994, Journal of experimental psychology. General.

[36]  Juliane M. Stopfer,et al.  Facebook Profiles Reflect Actual Personality, Not Self-Idealization , 2010, Psychological science.

[37]  Michael Luca Reviews, Reputation, and Revenue: The Case of Yelp.Com , 2016 .

[38]  Yasemin Besen‐Cassino Cool Stores Bad Jobs , 2013 .

[39]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[40]  P. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[41]  Beat Fasel,et al.  Automati Fa ial Expression Analysis: A Survey , 1999 .

[42]  V. Kshirsagar,et al.  Face recognition using Eigenfaces , 2011, 2011 3rd International Conference on Computer Research and Development.

[43]  Kiyoharu Aizawa,et al.  Empirical Observation of User Activities: Check-ins, Venue Photos and Tips in Foursquare , 2014, WISMM '14.