Emotional and Social Signals: A Neglected Frontier in Multimedia Computing?

The role of emotional and social signals in multimedia has not been a core concern of the multimedia research community--an omission explored at the 22nd ACM International Conference on Multimedia during a panel titled, "Emotional and Social Signals in Multimedia: Where Art Thou?" The panel discussion revealed major gaps in the formulation, understanding, and application of emotional and social signal processing in the multimedia domain. Here, the authors review the challenges in bringing this new domain to multimedia, summarizing current feelings in the research community based on discussions during the panel.

[1]  Mohan S. Kankanhalli,et al.  Temporal encoded F-formation system for social interaction detection , 2013, ACM Multimedia.

[2]  Björn W. Schuller,et al.  Categorical and dimensional affect analysis in continuous input: Current trends and future directions , 2013, Image Vis. Comput..

[3]  Hatice Gunes,et al.  Automatic Prediction of Perceived Traits Using Visual Cues under Varied Situational Context , 2014, 2014 22nd International Conference on Pattern Recognition.

[4]  J. Russell.  A circumplex model of affect. , 1980 .

[5]  Hatice Gunes,et al.  Automatic analysis of facial attractiveness from video , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[6]  Rosalind W. Picard.  Affective media and wearables: surprising findings , 2014, ACM Multimedia.

[7]  Maja Pantic,et al.  Implicit human-centered tagging [Social Sciences] , 2009, IEEE Signal Process. Mag..

[8]  Hatice Gunes,et al.  Automatic, Dimensional and Continuous Emotion Recognition , 2010, Int. J. Synth. Emot..

[9]  Rana El Kaliouby,et al.  Automatic measurement of ad preferences from facial responses gathered over the Internet , 2014, Image Vis. Comput..

[10]  Ioannis Patras,et al.  Fusion of facial expressions and EEG for implicit affective tagging , 2013, Image Vis. Comput..

[11]  P. Ekman,et al.  Unmasking the face : a guide to recognizing emotions from facial clues , 1975 .

[12]  Gwenn Englebienne,et al.  Mining for motivation: using a single wearable accelerometer to detect people's interests , 2012, IMMPD '12.

[13]  British Columbia,et al.  COMPASSIONATE WRATH: TRANSPERSONAL APPROACHES TO ANGER , 2000 .

[14]  Maja Pantic,et al.  Is this joke really funny? judging the mirth by audiovisual laughter analysis , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[15]  Alessandro Perina,et al.  Unveiling the multimedia unconscious: implicit cognitive processes and multimedia content analysis , 2013, ACM Multimedia.

[16]  Mohammad Soleymani,et al.  Continuous emotion detection in response to music videos , 2011, Face and Gesture 2011.

[17]  Alex Pentland,et al.  Honest Signals - How They Shape Our World , 2008 .

[18]  Anita Sharma,et al.  Personality and Patterns of Facebook Usage , 2016 .

[19]  Jiebo Luo,et al.  Understanding Kin Relationships in a Photo , 2012, IEEE Transactions on Multimedia.

[20]  K. Scherer.  What are emotions? And how can they be measured? , 2005 .

[21]  Daniel Gatica-Perez,et al.  The YouTube Lens: Crowdsourced Personality Impressions and Audiovisual Analysis of Vlogs , 2013, IEEE Transactions on Multimedia.

[22]  Daniel Gatica-Perez,et al.  Automatic nonverbal analysis of social interaction in small groups: A review , 2009, Image Vis. Comput..

[23]  Alessandro Vinciarelli,et al.  Automatic personality perception: Prediction of trait attribution based on prosodic features extended abstract , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[24]  Jean Carletta,et al.  The AMIDA Automatic Content Linking Device: Just-in-Time Document Retrieval in Meetings , 2008, MLMI.

[25]  Daniel Gatica-Perez,et al.  Cross-domain personality prediction: from video blogs to small group meetings , 2013, ICMI '13.

[26]  Yi Zhu,et al.  Alone or together: measuring users' viewing experience in different social contexts , 2014, Electronic Imaging.