Context-Aware Multimodal Sharing of Emotions

Computer mediated interaction often lacks of expressivity, in particular for emotion communication. Therefore, we present a concept for context-aware multimodal sharing of emotions for human-to-computer-to-human interaction in social networks. The multimodal inputs and outputs of this system are distributed in a smart environment in order to grant a more immersive and natural interaction experience. The context information is used to improve the opportuneness and the quality of feedback. We implemented an evaluation scenario and we conducted an observation study during some events with the participants. We reported our considerations at the end of this paper.

[1]  Rosalind W. Picard.  Affective computing: (526112012-054) , 1997 .

[2]  Ana Paiva,et al.  Affective Interactions: Toward a New Generation of Computer Interfaces? , 2000, IWAI.

[3]  S. Dornbush,et al.  XPOD – A Human Activity and Emotion Aware Mobile Music Player , 2005, 2005 2nd Asia Pacific Conference on Mobile Technology, Applications and Systems.

[4]  Rosalind W. Picard,et al.  Affective intelligence — the missing link? , 1997 .

[5]  Alladi Venkatesh,et al.  Has the Internet become indispensable? , 2004, CACM.

[6]  Roshan Lalintha Peiris,et al.  Huggy Pajama: a mobile parent and child hugging communication system , 2008, IDC.

[7]  Didier Perroud,et al.  Context-Based Generation of Multimodal Feedbacks for Natural Interaction in Smart Environments , 2012 .

[8]  Mor Naaman,et al.  Network properties and social sharing of emotions in social awareness streams , 2011, CSCW.

[9]  Ewald Moser,et al.  Amygdala activation at 3T in response to human and avatar facial expressions of emotions , 2007, Journal of Neuroscience Methods.

[10]  C. Nass,et al.  Emotion in human-computer interaction , 2002 .

[11]  David Matsumoto,et al.  Nonverbal Behaviors and Cross-Cultural Communication in the New Era , 2013 .

[12]  Lijun Yin,et al.  Static and dynamic 3D facial expression recognition: A comprehensive survey , 2012, Image Vis. Comput..

[13]  Riccardo Leonardi,et al.  A Connotative Space for Supporting Movie Affective Recommendation , 2011, IEEE Transactions on Multimedia.

[14]  Rosalind W. Picard.  Affective Computing: From Laughter to IEEE , 2010 .

[15]  Cynthia LeRouge,et al.  Developing multimodal intelligent affective interfaces for tele-home health care , 2003, Int. J. Hum. Comput. Stud..

[16]  Orville Leverne Clubb.  Human-to-Computer-to-Human Interactions (HCHI) of the communications revolution , 2007, INTR.

[17]  Theodoros Iliou,et al.  Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011 , 2012, Artificial Intelligence Review.

[18]  Jun Rekimoto,et al.  Smiling makes us happier: enhancing positive mood and communication with smile-encouraging digital appliances , 2011, UbiComp '11.

[19]  Eva Hudlicka,et al.  To feel or not to feel: The role of affect in human-computer interaction , 2003, Int. J. Hum. Comput. Stud..

[20]  Rosalind W. Picard.  Affective Computing , 1997 .

[21]  Tony Stockman,et al.  BCS-HCI '12 Proceedings of the 26th Annual BCS Interaction Specialist Group Conference on People and Computers, BCS-HCI 2012, 12-14 September 2012, Birmingham, UK , 2012, BCS HCI.

[22]  Sharon L. Oviatt,et al.  Multimodal Interfaces: A Survey of Principles, Models and Frameworks , 2009, Human Machine Interaction.

[23]  Tanja Schultz,et al.  Towards emotion recognition from electroencephalographic signals , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[24]  R. Plutchik.  The Nature of Emotions , 2001 .

[25]  Denis Lalanne,et al.  Human Machine Interaction, Research Results of the MMI Program , 2009, Human Machine Interaction.

[26]  Daniel McDuff,et al.  Crowdsourcing Facial Responses to Online Videos , 2012, IEEE Transactions on Affective Computing.

[27]  Hugo Silva,et al.  Multimodal biosignal sensor data handling for emotion recognition , 2011, 2011 IEEE SENSORS Proceedings.

[28]  P. Philippot,et al.  Social sharing of emotion : new evidence and new questions , 1998 .

[29]  Nicu Sebe,et al.  Affective multimodal human-computer interaction , 2005, ACM Multimedia.

[30]  Andrea Kleinsmith,et al.  Affective Body Expression Perception and Recognition: A Survey , 2013, IEEE Transactions on Affective Computing.

[31]  Marko Munih,et al.  Emotion-aware system for upper extremity rehabilitation , 2009, 2009 Virtual Rehabilitation International Conference.

[32]  Julie A. Jacko.  Ambient, ubiquitous and intelligent interaction , 2009 .

[33]  Stéphane Pierroz,et al.  Generic Framework for Transforming Everyday Objects into Interactive Surfaces , 2009, HCI.

[34]  Rongrong Wang,et al.  Keep in touch: channel, expectation and experience , 2012, CHI.

[35]  Tzu-Wei Tsai,et al.  An affective computing approach to develop the game-based adaptive learning material for the elementary students , 2012, HCCE '12.

[36]  Christian Martyn Jones,et al.  Emotishare: emotion sharing on mobile devices , 2012, BCS HCI.