Collective Smile: Measuring Societal Happiness from Geolocated Images

The increasing adoption of social media provides unprecedented opportunities to gain insight into human nature at vastly broader scales. Regarding the study of population-wide sentiment, prior research commonly focuses on text-based analyses and ignores a treasure trove of sentiment-laden content: images. In this paper, we make methodological and computational contributions by introducing the Smile Index as a formalized measure of societal happiness. Detecting smiles in 9 million geo-located tweets over 16 months, we validate our Smile Index against both text-based techniques and self-reported happiness. We further make observational contributions by applying our metric to explore temporal trends in sentiment, relate public mood to societal events, and predict economic indicators. Reflecting upon the innate, language-independent aspects of facial expressions, we recommend future improvements and applications to enable robust, global-level analyses. We conclude with implications for researchers studying and facilitating the expression of collective emotion through socio-technical systems.

[1]  Munmun De Choudhury,et al.  "Narco" emotions: affect and desensitization in social media during the mexican drug war , 2014, CHI.

[2]  David Watson,et al.  Diurnal variation in the Positive Affects , 1989 .

[3]  Adam D. I. Kramer.  An unobtrusive behavioral model of "gross national happiness" , 2010, CHI.

[4]  Cindy K. Chung,et al.  The development and psychometric properties of LIWC2007 , 2007 .

[5]  Judith A. Hall,et al.  Status, Gender, and Nonverbal Behavior in Candid and Posed Photographs: A Study of Conversations Between University Employees , 2001 .

[6]  Mor Naaman,et al.  Network properties and social sharing of emotions in social awareness streams , 2011, CSCW.

[7]  Marie-Francine Moens,et al.  A machine learning approach to sentiment analysis in multilingual Web texts , 2009, Information Retrieval.

[8]  César A. Hidalgo,et al.  The Collaborative Image of The City: Mapping the Inequality of Urban Perception , 2013, PloS one.

[9]  Eric Horvitz,et al.  Major life changes and behavioral markers in social media: case of childbirth , 2013, CSCW.

[10]  E. Diener,et al.  Subjective well-being. , 1984, Psychological bulletin.

[11]  小紫 雅史,et al.  プロジェクトKの視点から見た米国行政事情(最終回)Gross National Happiness , 2008 .

[12]  M. Erb,et al.  Are emotions contagious? Evoked emotions while viewing emotionally expressive faces: quality, quantity, time course and gender differences , 2001, Psychiatry Research.

[13]  Philip Resnik,et al.  Holistic Sentiment Analysis Across Languages: Multilingual Supervised Latent Dirichlet Allocation , 2010, EMNLP.

[14]  J. Pennebaker,et al.  Linguistic Markers of Psychological Change Surrounding September 11, 2001 , 2004, Psychological science.

[15]  D. Keltner,et al.  PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES Expressions of Positive Emotion in Women ' s College Yearbook Pictures and Their Relationship to Personality and Life Outcomes Across Adulthood , 2004 .

[16]  Charles Tijus,et al.  Zygomatic Smile Detection: The Semi-Supervised Haar Training of a Fast and Frugal System: A Gift to OpenCV Community , 2010, RIVF.

[17]  Andrew Clark,et al.  Happiness, Habits and High Rank: Comparisons in Economic and Social Life , 2012, SSRN Electronic Journal.

[18]  Mor Naaman,et al.  Robust detection of hyper-local events from geotagged social media data , 2013, MDMKDD '13.

[19]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[20]  Gilad Mishne,et al.  Capturing Global Mood Levels using Blog Posts , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[21]  P. Ekman,et al.  DIFFERENCES Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion , 2004 .

[22]  P. Ekman,et al.  Voluntary facial action generates emotion-specific autonomic nervous system activity. , 1990, Psychophysiology.

[23]  Brendan T. O'Connor,et al.  From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series , 2010, ICWSM.

[24]  D. Sauter,et al.  Commonalities outweigh differences in the communication of emotions across human cultures [Letter to the editor] , 2013 .

[25]  D Winton.  Lottery of life. , 2000, Nursing standard (Royal College of Nursing (Great Britain) : 1987).

[26]  M. Hughes,et al.  The continuing significance of race revisited : A study of race, class, and quality of life in America, 1972 to 1996 , 1998 .

[27]  Rainer Lienhart,et al.  Empirical Analysis of Detection Cascades of Boosted Classifiers for Rapid Object Detection , 2003, DAGM-Symposium.

[28]  James E. Coverdill,et al.  Race, Ethnicity and the Quality of Life in America, 1972-2008 , 2011 .

[29]  U. Hess,et al.  Who to whom and why--Cultural differences and similarities in the function of smiles. , 2002 .

[30]  Amit P. Sheth,et al.  Cursing in English on twitter , 2014, CSCW.

[31]  Daniele Quercia,et al.  Tracking "gross community happiness" from tweets , 2012, CSCW.

[32]  Javier Hernandez,et al.  Mood meter: counting smiles in the wild , 2012, UbiComp.

[33]  M. Lorr,et al.  Two EITS Manual for the Profile of Mood States (1971 & 1992) , 1971 .

[34]  Gwen Littlewort,et al.  Toward Practical Smile Detection , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Yutaka Matsuo,et al.  Earthquake shakes Twitter users: real-time event detection by social sensors , 2010, WWW '10.

[36]  Josep Blat,et al.  Digital Footprinting: Uncovering Tourists with User-Generated Content , 2008, IEEE Pervasive Computing.

[37]  R. Larsen,et al.  The Satisfaction with Life Scale , 1985, Journal of personality assessment.

[38]  J. Russell.  Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. , 1994, Psychological bulletin.

[39]  Eric Horvitz,et al.  Characterizing and predicting postpartum depression from shared facebook data , 2014, CSCW.

[40]  P. Ekman.  An argument for basic emotions , 1992 .

[41]  A. Clark,et al.  Relative Income, Happiness and Utility: An Explanation for the Easterlin Paradox and Other Puzzles , 2007 .

[42]  John M. Neale,et al.  Prospective and cross-sectional mood reports offer no evidence of a "blue Monday" phenomenon. , 1985 .

[43]  Johan Bollen,et al.  Modeling Public Mood and Emotion: Twitter Sentiment and Socio-Economic Phenomena , 2009, ICWSM.

[44]  Jiebo Luo,et al.  Aesthetics and Emotions in Images , 2011, IEEE Signal Processing Magazine.

[45]  Christopher M. Danforth,et al.  Temporal Patterns of Happiness and Information in a Global Social Network: Hedonometrics and Twitter , 2011, PloS one.

[46]  M. Bradley,et al.  Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings , 1999 .

[47]  Dacher Keltner,et al.  Ekman, emotional expression, and the art of empirical epiphany , 2004 .

[48]  Oliver G. B. Garrod,et al.  Facial expressions of emotion are not culturally universal , 2012, Proceedings of the National Academy of Sciences.

[49]  Thomas R. Peterson,et al.  Effects of Self-Generated Facial Expressions on Mood , 1998 .

[50]  Daniele Quercia.  Don't worry, be happy: the geography of happiness on Facebook , 2013, WebSci.

[51]  M. Hughes,et al.  The continuing significance of race: a study of race, class, and quality of life in America, 1972-1985. , 1986, American sociological review.

[52]  Rafael Di Tella,et al.  Some Uses of Happiness Data in Economics , 2006 .

[53]  P. Ekman,et al.  Facial action coding system: a technique for the measurement of facial movement , 1978 .

[54]  Bryan C. Semaan,et al.  Hybrid media consumption: how tweeting during a televised political debate influences the vote decision , 2014, CSCW.

[55]  A. Logunov,et al.  Characteristics and predictability of Twitter sentiment series , 2011 .

[56]  Scott A. Golder,et al.  Diurnal and Seasonal Mood Vary with Work, Sleep, and Daylength Across Diverse Cultures , 2011 .

[57]  Mor Naaman,et al.  CityBeat: real-time social media visualization of hyper-local city data , 2014, WWW.

[58]  Jun Rekimoto,et al.  Smiling makes us happier: enhancing positive mood and communication with smile-encouraging digital appliances , 2011, UbiComp '11.

[59]  M. Lorr,et al.  Profile of mood states , 1971 .

[60]  Denise C. Park,et al.  Cultural influences on Facebook photographs. , 2013, International journal of psychology : Journal international de psychologie.

[61]  Shigehiro Oishi,et al.  Intensity of Smiling in Facebook Photos Predicts Future Life Satisfaction , 2012 .

[62]  P. Converse,et al.  The Quality of American Life: Perceptions, Evaluations, and Satisfactions , 1976 .

[63]  R. Nisbett,et al.  Culture and Aesthetic Preference: Comparing the Attention to Context of East Asians and Americans , 2008, Personality & social psychology bulletin.

[64]  Hugo Liu,et al.  A Corpus-based Approach to Finding Happiness , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[65]  Hillary Anger Elfenbein,et al.  On the universality and cultural specificity of emotion recognition: a meta-analysis. , 2002, Psychological bulletin.

[66]  Andrew V. Papachristos,et al.  More Coffee, Less Crime? The Relationship between Gentrification and Neighborhood Crime Rates in Chicago, 1991 to 2005 , 2011 .

[67]  N. Christakis,et al.  Dynamic spread of happiness in a large social network : longitudinal analysis of the Framingham Heart Study social network. Commentary , 2009 .

[68]  T H Monk,et al.  Circadian rhythms in human performance and mood under constant conditions , 1997, Journal of sleep research.

[69]  Christopher M. Danforth,et al.  Measuring the Happiness of Large-Scale Written Expression: Songs, Blogs, and Presidents , 2010, ArXiv.

[70]  Christina Kotchemidova.  Why We Say “Cheese”: Producing the Smile in Snapshot Photography , 2005 .

[71]  A. Ferrer-i-Carbonell.  Income and Well-being , 2002 .

[72]  Marguerite Gardiner,et al.  The Lottery of Life. , 2009 .